Flight trajectory replay method and aircraft

Abstract

An embodiment of the present invention provides a flight trajectory replay method and an aircraft. The method comprises: an aircraft capturing first images and collecting flight status data at multiple flight moments, the flight status data including at least one of position information, velocity information, and acceleration information; the trajectory of the aircraft at the multiple flight moments is determined based on detected stick manipulations of the aircraft; the aircraft generates a first trajectory based on the flight status data collected at the multiple flight moments, the first trajectory being smoother than the trajectory of the aircraft at the multiple flight moments; and the aircraft flies according to the generated first trajectory. The corrected first trajectory of the embodiment of the present invention can, as closely as possible, recreate the trajectory of the flight under human control.

Description

Technical Field

[0001] The present invention relates to the technical field of aircraft control, and in particular to a flight trajectory replay method and an aircraft. Background Art

[0002] With the development of drone technology, multi-rotor drones are becoming increasingly popular and are widely used in aerial photography, which can be used for disaster assessment, emergency rescue, on-site reconnaissance, military exercises, and other fields. Currently, the operation of commonly used small drones mainly relies on the pilot's handheld remote control, which makes their use relatively difficult. Pilots need to have certain flight techniques and theoretical knowledge to effectively use small drones, which limits their application. In film shooting scenes, pilots are required to fly the same trajectory as many times as possible. However, due to human factors, even professional pilots find it difficult to maintain the same trajectory over multiple flights.

[0003] In this context, how to make the UAV fly autonomously and achieve nearly consistent trajectories over multiple flights is a technical problem that technicians in this field are currently studying. Summary of the Invention

[0004] In view of this, an embodiment of the present invention provides a flight trajectory replay method and an aircraft, wherein the obtained optimized first trajectory can restore the flight trajectory during human control as closely as possible.

[0005] A first aspect of an embodiment of the present invention provides a method for replaying a flight trajectory, the method comprising:

[0006] The aircraft captures first images and collects flight status data at multiple flight moments, the flight status data including at least one of position information, velocity information, and acceleration information, wherein a trajectory of the aircraft at the multiple flight moments is determined based on detected stick operations of the aircraft;

[0007] The aircraft generates a first trajectory based on the flight status data collected at the multiple flight moments, wherein the smoothness of the first trajectory is higher than the smoothness of the trajectory of the aircraft at the multiple flight moments;

[0008] The aircraft flies according to the generated first trajectory.

[0009] A second aspect of an embodiment of the present invention provides an aircraft, comprising a processor, a memory, and a camera, wherein the memory is configured to store program instructions, and the processor is configured to call the program instructions to perform the following operations:

[0010] capturing first images and collecting flight status data at a plurality of flight moments, the flight status data including at least one of position information, velocity information, and acceleration information, wherein a flight trajectory of the aircraft at the plurality of flight moments is determined based on detected stick manipulations of the aircraft;

[0011] generating a first trajectory based on the flight status data collected at the plurality of flight moments, wherein the first trajectory has a smoothness higher than a smoothness of the trajectory of the aircraft at the plurality of flight moments;

[0012] The aircraft flies according to the generated first trajectory. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments of the present invention or the background technology.

[0014] Figure 1 1 is a schematic structural diagram of a flight control system according to an embodiment of the present invention;

[0015] Figure 2 This is a flow chart of a flight trajectory replay method according to an embodiment of the present invention;

[0016] Figure 3 Flowchart of another flight trajectory replay method according to an embodiment of the present invention;

[0017] Figure 4 This is a schematic structural diagram of an aircraft according to an embodiment of the present invention;

[0018] Figure 5 This is a schematic structural diagram of another aircraft according to an embodiment of the present invention;

[0019] Figure 6 This is a schematic structural diagram of another aircraft according to an embodiment of the present invention;

[0020] Figure 7 This is a schematic structural diagram of another aircraft according to an embodiment of the present invention. DETAILED DESCRIPTION

[0021] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without creative work are within the scope of protection of the present invention.

[0022] See Figure 1 , Figure 1This is a flight control system according to an embodiment of the present invention. The system includes an aircraft 101, a gimbal 102 mounted on the aircraft 101, and a ground control device 103 for controlling the aircraft or simultaneously controlling the aircraft 101 and the gimbal 102. The aircraft can typically be various types of unmanned aerial vehicles (UAVs), such as quad-rotor UAVs, hexacopter UAVs, etc. The gimbal 102 mounted on the aircraft can be a three-axis gimbal, that is, the attitude of the gimbal 102 can be controlled along three axes: pitch, roll, and yaw, so as to determine the orientation of the gimbal 102, so that the camera configured on the gimbal 102 can complete tasks such as aerial photography during flight.

[0023] The aircraft 101 can communicate (e.g., wirelessly) with a ground control device 103. The ground control device 103 can be a controller with a joystick, which controls the aircraft by adjusting the joystick. The ground control device 103 can also be a smart device such as a smartphone or tablet computer, which can control the automatic flight of the drone 101 by configuring a flight trajectory on a user interface (UI), or by controlling the automatic flight of the drone 101 through somatosensory control or other methods.

[0024] See also Figure 2 , Figure 2 The present invention provides a method for replaying a flight trajectory. Figure 1 The method is implemented based on the flight system shown in the figure, and can also be implemented based on other architectures; the method includes but is not limited to the following steps:

[0025] Step S201: The aircraft captures first images and collects flight status data at multiple flight moments.

[0026] Specifically, the aircraft may remain in the air for a period of time. The aircraft's flight during this period may be controlled by remote control. The multiple flight times may be portions of this period. The strategy used to select the multiple flight times from this period is not specified herein. Optionally, the multiple flight times are determined based on at least one of flight speed, stick operation, and a preset distance threshold while the aircraft is flying along the first trajectory. The following example illustrates this:

[0027] The first is that the aircraft will use each time it detects a stick operation on the aircraft during flight as a flight time. It is understandable that the aircraft is usually controlled by a control terminal (for example, a remote control, a mobile phone, or other terminal). The control terminal may receive a stick operation from the user on the joystick (virtual or physical) during the control of the aircraft. Accordingly, the aircraft will detect in real time whether there is a stick operation on the joystick. If there is, the time when the user's stick operation is detected will be used as a flight time, that is, each time a stick operation is detected will be used as a flight time. It is understandable that the moment when the user performs a stick operation on the joystick is usually a turning point in the aircraft's flight (that is, a key node for the change in flight status). Writing down these special moments can help to better restore the flight trajectory later.

[0028] The second method is to ensure that the aircraft's speed at each flight moment is no less than a preset speed threshold (for example, 0.1 m / s). This means that if the aircraft's speed is very low at a certain moment, the aircraft will not use that moment as the flight moment. This prevents the aircraft from using a moment with very little displacement change within a certain period of time as the flight moment. For example, when the aircraft's speed is very low upon initial control of the joystick, the flight moment will be determined as the flight moment. If the aircraft's displacement changes very little within a certain period of time, and the moment within that period is used as the flight moment, the subsequent restored flight trajectory will be uneven.

[0029] The third type is that the distance between any two adjacent flight moments of the aircraft is not less than a preset distance threshold. In a specific implementation, the aircraft can detect its own position in real time and record the current moment as the flight moment each time its distance changes by a preset distance. For example, assuming that the aircraft detects its own position at the first moment (starting moment), the second moment, the third moment, the fourth moment, the fifth moment, the sixth moment, the seventh moment, the eighth moment, and the ninth moment in chronological order; then, first determine the moment at which the distance relative to the first moment is not less than the preset distance threshold, and then use the determined moment as the flight moment. If the determined flight moment is the third moment, then use the third moment as the flight moment. Then, further determine the moment at which the distance relative to the third moment is not less than the preset threshold, and then use the determined moment as the flight moment. If the determined flight moment is the fourth moment, then use the fourth moment as the flight moment, and so on.

[0030] The aircraft will collect flight status data at each of the multiple flight moments, and the flight status data includes at least one of position information, speed information and acceleration information. Optionally, the flight status data includes the position information, optionally, the flight status data includes position information and acceleration information, optionally, the flight status data includes position information, acceleration information and speed information. For example, when the flight status data includes location, the location information can be specifically determined by the aircraft through at least one of a positioning system (for example, Global Positioning System (GPS), BeiDou Navigation Satellite System (BDS), Galileo satellite navigation system (GNSS), etc.), visual positioning technology and inertial measurement unit (IMU). Optionally, the aircraft records its own starting position of flight, and then detects the flight acceleration and flight direction in real time through the IMU, and then calculates the flight speed based on the flight acceleration, and then calculates the flight distance based on the flight time and flight speed, and finally determines the real-time position of the aircraft based on the flight distance, flight direction and starting position of flight. For another example, when the flight status data includes acceleration information, the acceleration information can be obtained by the aircraft through an acceleration sensor, or it can be calculated by the aircraft based on the change in speed through a pre-configured algorithm. For another example, when the flight status data includes speed information, the speed information may be calculated by the aircraft using a pre-configured algorithm based on changes in its own position.

[0031] In addition, the aircraft captures a first image at each of the multiple flight moments. The first images captured by the aircraft at different locations are different, while the first images captured by the aircraft at the same location are the same. Each first image captured at each flight moment can be associated with location information to indicate the location at which the image was captured. Optionally, the first image can be a grayscale image, an RGB image, or the like.

[0032] Step S202: The aircraft generates a first trajectory according to the flight status data collected at the multiple flight moments.

[0033] It is understood that the aircraft first generates a first trajectory based on flight status data collected at multiple flight moments. The first trajectory is roughly the same as the trajectory of the aircraft during the above period of time. For ease of understanding, an optional method for generating the first trajectory is provided below:

[0034] The flight status data includes position information P, velocity information V, and acceleration information A. The trajectory obtained by sequentially concatenating the flight sub-trajectories of the aircraft between any two adjacent flight moments is the first trajectory. The sub-trajectory of the aircraft between any two adjacent flight moments can be described by a fifth-order polynomial. Assuming that the earlier of the two adjacent flight moments is t0 and the later is t1, and the time interval between t1 and t0 is dt, the fifth-order polynomial is specifically as follows:

[0035]

[0036] In formula 1-1, c x0 ,……,c x5 is the coefficient of the x-axis trajectory polynomial, c y0 ,……,c y5 is the coefficient of the y-axis trajectory polynomial, c z0 ,……,c z5 is the coefficient of the z-axis trajectory polynomial, and t is the flight time between any two adjacent flight moments, with t taking values ​​in the interval [0, dt]. Next, we need to solve for the coefficients of the x-axis trajectory polynomial, the y-axis trajectory polynomial, and the z-axis trajectory polynomial. The principles for solving the coefficients of these three axial trajectory polynomials are the same, so the following example will describe the solution for the coefficient of the x-axis trajectory polynomial:

[0037] The optimization function between any two flight moments is to minimize the change in Jerk (the first derivative of acceleration) (minimum jerk). Based on the position, velocity, and acceleration information in the X-axis direction, an analytical expression for the trajectory coefficient in the X-axis direction is obtained. Assume that the flight state data S0 of the earlier flight moment between any two adjacent flight moments is S0 = {P0, V0, A0, t0}, where P0 is the position of the aircraft at the earlier flight moment between any two adjacent flight moments, and V0 is the velocity of the aircraft at the earlier flight moment between any two adjacent flight moments. A0 is the acceleration of the aircraft at the earlier of any two adjacent flight moments, and t0 is the earlier of any two adjacent flight moments; the flight status data S0 of the later of the two adjacent flight moments is S0 = {P1, V1, A1, t1}, where P1 is the position of the aircraft at the later of any two adjacent flight moments, V1 is the velocity of the aircraft at the later of any two adjacent flight moments, A1 is the acceleration of the aircraft at the later of any two adjacent flight moments, and t1 is the later of any two adjacent flight moments; then, the X-axis component of the flight status data S0 = {P0, V0, A0, t0} of the earlier of the two adjacent flight moments is S0 x ={P x0,V x0 ,A x0 ,t x0}, where P x0 Indicates the x-axis position, V x0 Indicates the speed in the x-axis direction, A x0 represents the acceleration in the x-axis direction, t x0 Indicates time; the flight status data S1 = {P1, V1, A1, t1} of the later flight time in any two adjacent flight times has a component in the X-axis direction of S1 x ={P x1 ,V x1 ,A x1 ,t x1}, where P x1 Indicates the x-axis position, V x1 Indicates the speed in the x-axis direction, A x1 represents the acceleration in the x-axis direction, t x0 represents time; the intermediate variable shown in formula 1-2 can be obtained:

[0038]

[0039] Based on formula 1-2, we can further obtain the intermediate variables shown in formula 1-3:

[0040]

[0041] Based on formula 1-3, the coefficients of the x-axis trajectory polynomial can be further obtained, as shown in formula 1-4:

[0042]

[0043] The above describes how to derive the analytical formula for the X-axis trajectory coefficients based on the X-axis position, velocity, and acceleration information. The same principle can be used to derive the analytical formulas for the Y-axis and Z-axis trajectory coefficients. This allows us to determine the sub-trajectory of the aircraft between any two adjacent flight moments using Formula 1-1. The first trajectory described above can then be obtained by sequentially concatenating the sub-trajectories of the aircraft between any two adjacent flight moments.

[0044] Step S203: The aircraft captures a second image while flying along the first trajectory.

[0045] Step S204: The aircraft compares the second image with the first image to determine an offset.

[0046] Specifically, for example, the aircraft compares the texture in the second image with the texture in the first image to determine the offset. More specifically, the aircraft determines the offset of the aircraft using a closed-loop detection method. The specific principle is as follows: a first image that is closest to the currently captured second image is selected from the large number of first images captured above according to a pre-configured image matching algorithm (for example, an image similarity comparison algorithm based on a bag-of-words model), the position of the aircraft when the closest first image was captured is then determined, and finally, the offset between the position of the aircraft on the first trajectory when the current second image was captured and the position of the aircraft when the closest first image was captured is determined. The determined offset is the offset of the aircraft.

[0047] It can be understood that, according to this principle, the aircraft can determine the offset at each moment during the flight along the first trajectory, and the offset P of the aircraft can be determined. offset (t) can also include the X-axis component The Y-axis component and the Z-axis component That is, the offset P offset (t) can be expressed as Step S205: The aircraft corrects the first trajectory according to the offset.

[0048] Specifically, the offset determined at the previous moment can be used to make corrections at the next moment. Since the first trajectory is obtained by sequentially splicing all the flight sub-trajectories between two adjacent flight moments on the first trajectory, the principle of using the offset to correct the flight sub-trajectories between any two adjacent flight moments is first described here; optionally, the correction principle can be to use the offset to correct the flight sub-trajectories between any two adjacent flight moments. Substitute into the above formula 1-1 to obtain the corrected sub-trajectory of the flight trajectory between any two adjacent flight moments, as shown in formula 2-1:

[0049]

[0050] The corrected sub-trajectories between two adjacent flight moments are combined to form the corrected first trajectory. Therefore, it can be assumed that the aircraft is actually flying along the corrected first trajectory. The principle by which the aircraft controls itself to fly along the corrected first trajectory can be as follows:

[0051] The aircraft determines flight state data at each control moment during flight along the corrected first trajectory (it is understood that the aircraft needs to continuously adjust its state so that the final flight trajectory is the corrected first trajectory, and each moment of state adjustment can be referred to as a control moment). The flight state data includes at least one of position information, velocity information, and acceleration information. The aircraft then adjusts its state to the state indicated by the flight state data at each control moment. In other words, the aircraft knows in real time the desired position, velocity, and acceleration. It then adjusts its actual position, velocity, and acceleration to achieve the desired position, velocity, and acceleration.

[0052] The following example describes how to obtain the desired position, flight speed, and flight acceleration of the aircraft:

[0053] Assume that the corrected first trajectory is composed of n corrected sub-trajectories. Each of these n corrected sub-trajectories is obtained by correcting the flight trajectory between two adjacent flight times (as described above). The corrected sub-trajectories obtained by correcting the flight trajectory between two adjacent flight times in different groups can be different. As shown in Formula 2-1, each corrected sub-trajectory is composed of a 5th-order polynomial with 6 coefficients. The coefficients and start and end times of these polynomials are stored in the following data structure:

[0054]

[0055] In formula 3-1, trajectory_x[n][6] stores the 6 coefficients of the n corrected sub-trajectories along the x-axis, a total of 6*n coefficients, trajectory_y[n][6] stores the 6 coefficients of the n corrected sub-trajectories along the y-axis, a total of 6*n coefficients, trajectory_z[n][6] stores the 6 coefficients of the n corrected sub-trajectories along the z-axis, a total of 6*n coefficients; trajectory_time[n+1] stores the end time of the n corrected sub-trajectories on the entire time axis. For example, the corrected first trajectory consists of 3 corrected sub-trajectories, and the elapsed time of these 3 corrected sub-trajectories is 1 second, 2 seconds, and 3 seconds respectively. Then on the time axis, the starting time of the corrected first trajectory is the 0th second, the end time of the first corrected sub-trajectory is the 1st second, the end time of the second corrected sub-trajectory is the 3rd second, and the end time of the third corrected sub-trajectory is the 6th second. 6s is also the time required to complete the flight of the corrected first trajectory.

[0056] Then, the aircraft executes the following process according to the data structure shown in 3-1:

[0057] 1. Set a global clock and reset the clock to 0 when you start flying along the corrected first trajectory. This 0 is actually the starting time of the corrected first trajectory on the timeline.

[0058] 2. Assume that the control frequency of the aircraft in the trajectory following process is frequency, that is, it sends frequency control instructions in one second, then the control instruction calculation period is 1 / frequency, that is, the control instruction is calculated every 1 / frequency seconds, so the time growth interval of the global clock is 1 / frequency (the clock continuously updates its own time according to the formula tick = tick + 1 / frequency). Each time tick of the clock corresponds to a desired position, velocity and acceleration on the corrected first trajectory. According to the current time tick of the clock, traverse trajectory_time[n+1] to find which corrected sub-trajectory the tick corresponds to. The corrected m-th sub-trajectory found satisfies the following formula:

[0059] trajectory_time[m]≤tick≤trajectory_time[m+1]3-2

[0060] 3. Calculate the current position p that needs to be reached based on the parameters of the modified m-th sub-trajectory desired , the speed v that needs to be achieved desired , and the acceleration a that needs to be achieved desired:

[0061] Since the sub-trajectory polynomial is normalized, the tick at this location must also be normalized, i.e.

[0062]

[0063] Among them, trajectory_time[m+1]-trajectory_time[m] represents the total elapsed time of the modified m-th sub-trajectory, and tick-trajectory_time[m] represents the time when the tick moment has entered the modified m-th sub-trajectory. Then, the required position p is calculated according to the trajectory polynomial shown in formula 2-1. desired , the speed v that needs to be achieved desired , the acceleration a that needs to be achieved desired , the calculation formulas for each quantity are as follows (trajectory_x[n][6] is abbreviated as tx[n][6]):

[0064] The position p that needs to be reached desired for:

[0065]

[0066] Formula 4-1 is the matrix form of the linear equations. After expansion, it is the equations in Formula 2-1. Taking the x-axis as an example, after expansion, the position p that the x-axis needs to reach can be obtained. desired_x =tx[m][0]+ tx[m][1]*t m 1 +tx[m][2]*t m 2 +tx[m][3]*t m 3 +tx[m][4]*t m 4 +tx[m][5]* t m 5 , where tx[m][0] is equal to the constant term of the modified m-th sub-trajectory polynomial In addition, the positions that need to be reached in the Y and Z axes can refer to the description of the X axis. Based on the positions of the X axis, the Y axis and the Z axis, the position p that needs to be reached can be obtained. desired .

[0067] The speed v that needs to be achieved desired for:

[0068]

[0069] Formula 4-2 is the matrix form of the linear equations. After expansion, it is the first-order derivative of the equations in Formula 3-1. Taking the x-axis as an example, after expansion, the speed v that needs to be achieved in the x-axis can be obtained. desired_x = tx[m][1]+2*tx[m][2]*t m 1 +3*tx[m][3]*t m 2 +4*tx[m][4]*t m 3 +5* tx[m][5]*t m 4 , where tx[m][1] is equal to the first-order term in the polynomial of the mth modified sub-trajectory; in addition, the speeds to be achieved in the Y-axis and Z-axis directions can refer to the description of the X-axis direction. Based on the speeds in the X-axis, Y-axis, and Z-axis directions, the speed v to be achieved can be obtained. desired .

[0070] The acceleration a required desired for:

[0071]

[0072] Formula 4-3 is the matrix form of the linear equations. After expansion, it is the second-order derivative of the equations in Formula 3-1. Taking the x-axis as an example, after expansion, the acceleration a required to be achieved in the x-axis direction can be obtained. desired_x =2* tx[m][2]+6*tx[m][3]*t m 1 +12*tx[m][4]*t m 2 +20*tx[m][5]*t m 3 , where tx[m][2] is equal to the second-order term of the mth modified sub-trajectory polynomial; in addition, the accelerations required in the Y-axis and Z-axis directions can refer to the description of the X-axis direction. Based on the accelerations in the X-axis, Y-axis, and Z-axis directions, the required acceleration a can be obtained. desired .

[0073] According to the above principle, we can get the position p that needs to be reached at each moment when flying on the mth corrected sub-trajectory desired , the speed v that needs to be achieved desired , the acceleration a that needs to be achieved desired Based on the same principle, we can deduce the position p that the aircraft needs to reach at each moment when flying on the entire corrected first trajectory. desired , the speed v that needs to be achieved desired, the acceleration a that needs to be achieved desired .

[0074] Accordingly, the position p that the aircraft needs to reach during the flight desired , the speed v that needs to be achieved desirsd , the acceleration a that needs to be achieved desired Substitute into the proportional-integral-derivative (PID) control algorithm (or other algorithms) to calculate the control command, which is used to instruct the aircraft to make corresponding adjustments so that the position of the aircraft needs to reach the position p desired , the speed reaches the required speed v desired , and the acceleration reaches the required acceleration a desired .

[0075] exist Figure 2 In the method shown, the aircraft collects flight status data and captures a first image via a camera during flight. A first trajectory is then generated using this flight status data. Subsequently, a second image is captured in real time while the aircraft is flying along the first trajectory. The second image is then compared with the first image to determine the aircraft's deviation from the first trajectory. This deviation is then used to correct the first trajectory, and flight continues along the corrected first trajectory. Because the generation of the corrected first trajectory takes into account the aircraft's deviation during flight, it can reduce iterative errors generated during flight, ensuring that the final first trajectory generated for autonomous flight closely replicates the human-controlled flight trajectory.

[0076] See also Figure 3 , Figure 3 The present invention provides a method for replaying a flight trajectory. Figure 1 The method is implemented based on the flight system shown in the figure, and can also be implemented based on other architectures; the method includes but is not limited to the following steps:

[0077] Step S301: During the flight of the aircraft, a plurality of flight moments are determined according to at least one of a flight speed, a stick operation, and a preset distance threshold, and flight status data of the aircraft at the plurality of flight moments are collected.

[0078] Specifically, optionally, the flight of the aircraft during the period of time may be controlled by human remote control. The strategy for selecting the multiple flight times from the period of time is not limited herein. Several possible implementation solutions are listed below:

[0079] The first is that the aircraft will use each time it detects a stick operation on the aircraft during flight as a flight time. It is understandable that the aircraft is usually controlled by a control terminal (for example, a remote control, a mobile phone, or other terminal). The control terminal may receive a stick operation from the user on the joystick (virtual or physical) during the control of the aircraft. Accordingly, the aircraft will detect in real time whether there is a stick operation on the joystick. If there is, the time when the user's stick operation is detected will be used as a flight time, that is, each time a stick operation is detected will be used as a flight time. It is understandable that the moment when the user performs a stick operation on the joystick is usually a turning point in the aircraft's flight (that is, a key node for the change in flight status). Writing down these special moments can help to better restore the flight trajectory later.

[0080] The second method is to ensure that the aircraft's speed at each flight moment is no less than a preset speed threshold (for example, 0.1 m / s). This means that if the aircraft's speed is very low at a certain moment, the aircraft will not use that moment as the flight moment. This prevents the aircraft from using a moment with very little displacement change within a certain period of time as the flight moment. For example, when the aircraft's speed is very low upon initial control of the joystick, the flight moment will be determined as the flight moment. If the aircraft's displacement changes very little within a certain period of time, and the moment within that period is used as the flight moment, the subsequent restored flight trajectory will be uneven.

[0081] The third type is that the distance between any two adjacent flight moments of the aircraft is not less than a preset distance threshold. In a specific implementation, the aircraft can detect its own position in real time and record the current moment as the flight moment each time its distance changes by a preset distance. For example, assuming that the aircraft detects its own position at the first moment (starting moment), the second moment, the third moment, the fourth moment, the fifth moment, the sixth moment, the seventh moment, the eighth moment, and the ninth moment in chronological order; then, first determine the moment at which the distance relative to the first moment is not less than the preset distance threshold, and then use the determined moment as the flight moment. If the determined flight moment is the third moment, then use the third moment as the flight moment. Then, further determine the moment at which the distance relative to the third moment is not less than the preset threshold, and then use the determined moment as the flight moment. If the determined flight moment is the fourth moment, then use the fourth moment as the flight moment, and so on.

[0082] The aircraft will continue to fly in the air for a period of time, and the multiple flight moments may be partial moments of the period of time. The UAV will collect flight status data at each of the multiple flight moments, and the flight status data includes at least one of position information, speed information and acceleration information. Optionally, the flight status data includes the position information, optionally, the flight status data includes position information and acceleration information, optionally, the flight status data includes position information, acceleration information and speed information. For example, when the flight status data includes position, the position information can be specifically determined by the aircraft through at least one of a positioning system (for example, Global Positioning System, GPS, BeiDou Navigation Satellite System, BDS, Galileo satellite navigation system, GNSS, etc.), visual positioning technology and inertial measurement unit (IMU). Optionally, the aircraft records its own starting position of flight, then uses the IMU to detect flight acceleration and flight direction in real time, calculates flight speed based on flight acceleration, and then calculates flight distance based on elapsed flight time and flight speed. Finally, the aircraft's real-time position is determined based on the flight distance, flight direction, and starting position. For another example, when the flight status data includes acceleration information, the acceleration information can be obtained by the aircraft through an acceleration sensor, or can be calculated by the aircraft based on changes in speed using a pre-configured algorithm. For another example, when the flight status data includes speed information, the speed information can be calculated by the aircraft based on changes in its own position using a pre-configured algorithm.

[0083] Step S302: the aircraft generates a first trajectory according to the flight status data collected by the aircraft at the multiple flight moments.

[0084] Specifically, the aircraft first generates a first trajectory based on flight status data collected at multiple flight moments. The first trajectory has a substantially similar direction to the trajectory of the aircraft during the aforementioned period of time. Optionally, the first trajectory is smoother than the trajectory of the aircraft during the aforementioned period of time. For ease of understanding, an optional method for generating the first trajectory is provided below:

[0085] The flight status data includes position information P, velocity information V, and acceleration information A. The trajectory obtained by sequentially concatenating the flight sub-trajectories of the aircraft between any two adjacent flight moments is the first trajectory. The sub-trajectory of the aircraft between any two adjacent flight moments can be described by a fifth-order polynomial. Assuming that the earlier of the two adjacent flight moments is t0 and the later is t1, and the time interval between t1 and t0 is dt, the fifth-order polynomial is specifically as follows:

[0086]

[0087] In formula 1-1, c x0 ,……,c x5 is the coefficient of the x-axis trajectory polynomial, c y0 ,……,c y5 is the coefficient of the y-axis trajectory polynomial, c z0 ,……,c z5 is the coefficient of the z-axis trajectory polynomial, and t is the flight time between any two adjacent flight moments, with t taking values ​​in the interval [0, dt]. Next, we need to solve for the coefficients of the x-axis trajectory polynomial, the y-axis trajectory polynomial, and the z-axis trajectory polynomial. The principles for solving the coefficients of these three axial trajectory polynomials are the same, so the following example will describe the solution for the coefficient of the x-axis trajectory polynomial:

[0088] The optimization function between any two flight moments is to minimize the change in Jerk (the first derivative of acceleration) (minimum jerk). Based on the position, velocity, and acceleration information in the X-axis direction, an analytical expression for the trajectory coefficient in the X-axis direction is obtained. Assume that the flight state data S0 of the earlier flight moment between any two adjacent flight moments is S0 = {P0, V0, A0, t0}, where P0 is the position of the aircraft at the earlier flight moment between any two adjacent flight moments, and V0 is the velocity of the aircraft at the earlier flight moment between any two adjacent flight moments. A0 is the acceleration of the aircraft at the earlier of any two adjacent flight moments, and t0 is the earlier of any two adjacent flight moments; the flight status data S0 of the later of the two adjacent flight moments is S0 = {P1, V1, A1, t1}, where P1 is the position of the aircraft at the later of any two adjacent flight moments, V1 is the velocity of the aircraft at the later of any two adjacent flight moments, A1 is the acceleration of the aircraft at the later of any two adjacent flight moments, and t1 is the later of any two adjacent flight moments; then, the X-axis component of the flight status data S0 = {P0, V0, A0, t0} of the earlier of the two adjacent flight moments is S0 x ={P x0,V x0 ,A x0 ,t x0}, where P x0 Indicates the x-axis position, V x0 Indicates the speed in the x-axis direction, A x0 represents the acceleration in the x-axis direction, t x0 Indicates time; the flight status data S1 = {P1, V1, A1, t1} of the later flight time in any two adjacent flight times has a component in the X-axis direction of S1 x ={P x1 ,V x1 ,A x1 ,t x1}, where P x1 Indicates the x-axis position, V x1 Indicates the speed in the x-axis direction, A x1 represents the acceleration in the x-axis direction, t x0 represents time; the intermediate variable shown in formula 1-2 can be obtained:

[0089]

[0090] Based on formula 1-2, we can further obtain the intermediate variables shown in formula 1-3:

[0091]

[0092] Based on formula 1-3, the coefficients of the x-axis trajectory polynomial can be further obtained, as shown in formula 1-4:

[0093]

[0094] The above describes how to derive the analytical formula for the X-axis trajectory coefficients based on the X-axis position, velocity, and acceleration information. The same principle can be used to derive the analytical formulas for the Y-axis and Z-axis trajectory coefficients. This allows us to determine the sub-trajectory of the aircraft between any two adjacent flight moments using Formula 1-1. The first trajectory described above can then be obtained by sequentially concatenating the sub-trajectories of the aircraft between any two adjacent flight moments.

[0095] Step S303: The aircraft flies according to the generated first trajectory.

[0096] In one optional solution, during flight, the aircraft determines multiple flight moments based on at least one of flight speed, stick operation, and a preset distance threshold, and collects flight status data for the aircraft at these multiple flight moments. Specifically, during flight, the aircraft determines multiple flight moments based on at least one of flight speed, stick operation, and a preset distance threshold, captures first images at these multiple flight moments, and collects flight status data for the aircraft. First images captured by the aircraft at different locations are different, while first images captured by the aircraft at the same location are the same. Position information can be associated with each first image captured at each flight moment to indicate the location at which the image was captured. Optionally, the first image can be a grayscale image, an RGB image, or the like.

[0097] In addition, the aircraft flies according to the generated first trajectory, which may be: the aircraft captures a second image while flying according to the first trajectory; the aircraft compares the second image with the first image to determine an offset; and the aircraft corrects the first trajectory according to the offset so that the aircraft continues to fly.

[0098] The specific principle of determining the offset can be as follows: according to a pre-configured image matching algorithm (for example, an image similarity comparison algorithm based on a bag-of-words model), a first image closest to the currently captured second image is selected from the large number of first images captured above, and then the position of the aircraft when the closest first image was captured is found, and finally the position of the aircraft on the first trajectory when the current second image was captured is determined, and the offset between the position of the aircraft when the closest first image was captured is determined. The determined offset is the offset of the aircraft. It can be understood that according to this principle, the aircraft can determine the offset at each moment during the flight along the first trajectory, and the offset P of the aircraft is determined. offset (t) can also include the X-axis component The Y-axis component and the Z-axis component That is, the offset P offset (t) can be expressed as Optionally, the offset determined at a previous moment can be used for correction at a later moment.

[0099] Since the first trajectory is obtained by sequentially splicing all the flight sub-trajectories between two adjacent flight moments on the first trajectory, the principle of using the offset to correct the flight sub-trajectories between any two adjacent flight moments is first described here; optionally, the correction principle can be to use the offset Substitute into the above formula 1-1 to obtain the corrected sub-trajectory of the flight trajectory between any two adjacent flight moments, as shown in formula 2-1:

[0100]

[0101] The corrected sub-trajectories of the flight trajectory between two adjacent flight moments are spliced ​​together to form a corrected first trajectory. Therefore, it can be assumed that the aircraft is actually flying along the corrected first trajectory. Controlling the aircraft to fly along the corrected first trajectory can specifically include the following operations: First, the aircraft determines flight status data for each control moment during flight based on the corrected first trajectory. This flight status data includes at least one of position information, velocity information, and acceleration information. Then, the aircraft adjusts its state at each control moment to the state indicated by the flight status data at that control moment. Specifically, the aircraft obtains in real time the required position, flight speed, and flight acceleration during flight along the corrected first trajectory. The aircraft then adjusts its current actual position, actual flight speed, and actual flight acceleration to ensure that the aircraft reaches the required position, flight speed, and flight acceleration. The following describes an example of how to determine the aircraft's current required position, flight speed, and flight acceleration:

[0102] Assume that the corrected first trajectory is composed of n corrected sub-trajectories. Each of these n corrected sub-trajectories is obtained by correcting the flight trajectory between two adjacent flight times (as described above). The corrected sub-trajectories obtained by correcting the flight trajectory between two adjacent flight times in different groups can be different. As shown in Formula 2-1, each corrected sub-trajectory is composed of a 5th-order polynomial with 6 coefficients. The coefficients and start and end times of these polynomials are stored in the following data structure:

[0103]

[0104] In formula 3-1, trajectory_x[n][6] stores the 6 coefficients of the n corrected sub-trajectories along the x-axis, a total of 6*n coefficients, trajectory_y[n][6] stores the 6 coefficients of the n corrected sub-trajectories along the y-axis, a total of 6*n coefficients, trajectory_z[n][6] stores the 6 coefficients of the n corrected sub-trajectories along the z-axis, a total of 6*n coefficients; trajectory_time[n+1] stores the end time of the n corrected sub-trajectories on the entire time axis. For example, the corrected first trajectory consists of 3 corrected sub-trajectories, and the elapsed time of these 3 corrected sub-trajectories is 1 second, 2 seconds, and 3 seconds respectively. Then on the time axis, the starting time of the corrected first trajectory is the 0th second, the end time of the first corrected sub-trajectory is the 1st second, the end time of the second corrected sub-trajectory is the 3rd second, and the end time of the third corrected sub-trajectory is the 6th second. 6s is also the time required to complete the flight of the corrected first trajectory.

[0105] Then, the aircraft executes the following process according to the data structure shown in 3-1:

[0106] 1. Set a global clock and reset the clock to 0 when you start flying along the corrected first trajectory. This 0 is actually the starting time of the corrected first trajectory on the timeline.

[0107] 2. Assume that the control frequency of the aircraft in the trajectory following process is frequency, that is, it sends frequency control instructions in one second, then the control instruction calculation period is 1 / frequency, that is, the control instruction is calculated every 1 / frequency seconds, so the time growth interval of the global clock is 1 / frequency (the clock continuously updates its own time according to the formula tick = tick + 1 / frequency). Each time tick of the clock corresponds to a desired position, velocity and acceleration on the corrected first trajectory. According to the current time tick of the clock, traverse trajectory_time[n+1] to find which corrected sub-trajectory the tick corresponds to. The corrected m-th sub-trajectory found satisfies the following formula:

[0108] trajectory_time[m]≤tick≤trajectory_time[m+1] 3-2

[0109] 3. Calculate the current position p that needs to be reached based on the parameters of the modified m-th sub-trajectory desired , the speed v that needs to be achieved desired , and the acceleration a that needs to be achieved desired:

[0110] Since the sub-trajectory polynomial is normalized, the tick at this location must also be normalized, i.e.

[0111]

[0112] Among them, trajectory_time[m+1]-trajectory_time[m] represents the total elapsed time of the modified m-th sub-trajectory, and tick-trajectory_time[m] represents the time when the tick moment has entered the modified m-th sub-trajectory. Then, the required position p is calculated according to the trajectory polynomial shown in formula 2-1. desired , the speed v that needs to be achieved desired , the acceleration a that needs to be achieved desired , the calculation formulas for each quantity are as follows (trajectory_x[n][6] is abbreviated as tx[n][6]):

[0113] The position p that needs to be reached desired for:

[0114]

[0115] Formula 4-1 is the matrix form of the linear equations. After expansion, it is the equations in Formula 2-1. Taking the x-axis as an example, after expansion, the position p that the x-axis needs to reach can be obtained. desired_ =tx[m][0]+ tx[m][1]*t m 1 +tx[m][2]*t m 2 +tx[m][3]*t m 3 +tx[m][4]*t m 4 +tx[m][5]* t m 5 , where tx[m][0] is equal to the constant term of the modified m-th sub-trajectory polynomial In addition, the positions that need to be reached in the Y and Z axes can refer to the description of the X axis. Based on the positions of the X axis, the Y axis and the Z axis, the position p that needs to be reached can be obtained. desired .

[0116] The speed v that needs to be achieved desired for:

[0117]

[0118] Formula 4-2 is the matrix form of the linear equations. After expansion, it is the first-order derivative of the equations in Formula 3-1. Taking the x-axis as an example, after expansion, the speed v that needs to be achieved in the x-axis can be obtained. desired_x = tx[m][1]+2*tx[m][2]*t m 1 +3*tx[m][3]*t m 2 +4*tx[m][4]*t m 3 +5* tx[m][5]*t m 4 , where tx[m][1] is equal to the first-order term in the polynomial of the mth modified sub-trajectory; in addition, the speeds to be achieved in the Y-axis and Z-axis directions can refer to the description of the X-axis direction. Based on the speeds in the X-axis, Y-axis, and Z-axis directions, the speed v to be achieved can be obtained. desired .

[0119] The acceleration a required desired for:

[0120]

[0121] Formula 4-3 is the matrix form of the linear equations. After expansion, it is the second-order derivative of the equations in Formula 3-1. Taking the x-axis as an example, after expansion, the acceleration a required to be achieved in the x-axis direction can be obtained. desired_x =2* tx[m][2]+6*tx[m][3]*t m 1 +12*tx[m][4]*t m 2 +20*tx[m][5]*t m 3 , where tx[m][2] is equal to the second-order term of the mth modified sub-trajectory polynomial; in addition, the accelerations required in the Y-axis and Z-axis directions can refer to the description of the X-axis direction. Based on the accelerations in the X-axis, Y-axis, and Z-axis directions, the required acceleration a can be obtained. desired .

[0122] According to the above principle, we can get the position p that needs to be reached at each moment when flying on the mth corrected sub-trajectory desired , the speed v that needs to be achieved desired , the acceleration a that needs to be achieved desired Based on the same principle, we can deduce the position p that the aircraft needs to reach at each moment when flying on the entire corrected first trajectory. desired , the speed v that needs to be achieved desired, the acceleration a that needs to be achieved desired .

[0123] Accordingly, the position p that the aircraft needs to reach during the flight desired , the speed v that needs to be achieved desired , the acceleration a that needs to be achieved desired Substitute into the proportional-integral-derivative (PID) control algorithm (or other algorithms) to calculate the control command, which is used to instruct the aircraft to make corresponding adjustments so that the position of the aircraft needs to reach the position p desired , the speed reaches the required speed v desired , and the acceleration reaches the required acceleration a desired .

[0124] exist Figure 3 In the illustrated method, during flight, the aircraft determines multiple flight moments based on at least one of flight speed, stick control, and a preset distance threshold. Flight status data is then collected at these multiple determined flight moments, and a first trajectory is generated based on the flight status data. Finally, the aircraft flies along this first trajectory. Because the flight moments are determined based on at least one of flight speed, stick control, and the preset distance threshold, the majority of the determined flight moments are critical moments in the aircraft's flight. Therefore, the first trajectory, determined based on the flight status data collected at these flight moments, can more accurately reconstruct the human-controlled flight trajectory.

[0125] The following describes the pan-tilt platform follow-up control device and the control equipment according to the embodiment of the present invention.

[0126] See Figure 4 , Figure 4 4 is a schematic structural diagram of an aircraft 40 according to an embodiment of the present invention. The aircraft 40 includes a collection module 401, a generation module 402, a first shooting module 403, a first determination module 404 and an optimization module 405. Each module is described as follows.

[0127] The acquisition module 401 is used to capture first images and collect flight status data at multiple flight moments, where the flight status data includes at least one of position information, speed information, and acceleration information;

[0128] The generating module 402 is configured to generate a first trajectory based on the flight status data collected at the plurality of flight moments;

[0129] The first shooting module 403 is used to shoot a second image during the flight along the first trajectory;

[0130] The first determining module 404 is configured to compare the second image with the first image to determine an offset;

[0131] The optimization module 405 is configured to modify the first trajectory according to the offset so that the aircraft can continue to fly.

[0132] In an optional solution, the first determining unit 404 compares the second image with the first image to determine the offset. Specifically, the aircraft compares the texture in the second image with the texture in the first image to determine the offset.

[0133] In yet another optional solution, the aircraft further includes a second determining module and an adjusting module:

[0134] The second determining module is configured to determine, after the generating module 402 generates the first trajectory based on the flight status data collected at the plurality of flight moments, the flight status data at each control moment during the flight process based on the first trajectory, the flight status data including at least one of position information, velocity information, and acceleration information;

[0135] The adjustment module is used to adjust its own state at each control moment to the state indicated by the flight state data at each control moment.

[0136] In another optional solution, the location information is determined based on at least one of GPS technology, visual angle positioning technology and inertial sensor positioning technology.

[0137] In yet another optional solution, the aircraft further includes a third determination module configured to determine the multiple flight moments during flight based on at least one of flight speed, stick operation, and a preset distance threshold before the acquisition module captures the first image and collects flight status data at the multiple flight moments.

[0138] In another optional solution, the third determination module determines the multiple flight times according to at least one of a flight speed, a stick operation, and a preset distance threshold during the flight of the aircraft, specifically:

[0139] During the flight of the aircraft, each time a stick operation is detected for the aircraft is used as the flight time.

[0140] In another optional solution, the flight speed of the aircraft at each flight moment is not lower than a preset speed threshold.

[0141] In another optional solution, the distance between any two temporally adjacent flight moments of the aircraft is not less than a preset distance threshold.

[0142] It should be noted that the implementation of each module can also refer to Figure 2 The corresponding description of the method embodiment shown.

[0143] exist Figure 4 In the described aircraft 40, during flight, the aircraft collects flight status data and captures a first image via a camera. A first trajectory is then generated using this flight status data. Subsequently, while flying along the first trajectory, a second image is captured in real time. The second image is then compared with the first image to determine the offset of the aircraft while flying along the first trajectory. This offset is then used to correct the first trajectory, and flight continues along the corrected first trajectory. Because the generation of the corrected first trajectory takes into account the offset of the aircraft during flight, it is possible to reduce iteration errors generated during flight, allowing the final first trajectory generated for autonomous flight to closely replicate the human-controlled flight trajectory.

[0144] See Figure 5 , Figure 5 1 is a schematic structural diagram of an aircraft 50 according to an embodiment of the present invention. The aircraft 50 includes a determination module 501, a generation module 502, and a flight module 503. Each module is described as follows.

[0145] The determination module 501 is configured to determine multiple flight moments during flight of the aircraft based on at least one of flight speed, stick operation, and a preset distance threshold, and collect flight status data of the aircraft at the multiple flight moments, the flight status data including at least one of position information, speed information, and acceleration information;

[0146] The generating module 502 is configured to generate a first trajectory based on the flight status data collected by the aircraft at the plurality of flight moments;

[0147] The flight module 503 is configured to fly according to the generated first trajectory.

[0148] In an optional solution, the first determining module 501 determines multiple flight moments during the flight of the aircraft based on at least one of the flight speed, the stick operation, and a preset distance threshold, and collects flight status data of the aircraft at the multiple flight moments, specifically:

[0149] During the flight of the aircraft, each time a stick operation is detected for the aircraft is used as the flight time, and the flight status data of the aircraft at the flight time is collected.

[0150] In an optional solution, the flight speed of the aircraft at each flight moment is not lower than a preset speed threshold.

[0151] In another optional solution, the distance between any two temporally adjacent flight moments of the aircraft is not less than a preset distance threshold.

[0152] In another optional solution, the location information is determined based on at least one of GPS technology, visual angle positioning technology and inertial sensor positioning technology.

[0153] In another optional solution, during the flight of the aircraft, the determination module 501 determines multiple flight moments based on at least one of the flight speed, the stick operation, and a preset distance threshold, and collects flight status data of the aircraft at the multiple flight moments, specifically:

[0154] During the flight, a plurality of flight moments are determined based on at least one of a flight speed, a stick operation, and a preset distance threshold, and a first image is captured and flight status data of the aircraft is collected at the plurality of flight moments;

[0155] The flight module 503 flies according to the generated first trajectory, specifically:

[0156] First, capturing a second image during flight along the first trajectory;

[0157] Then, comparing the second image with the first image to determine an offset;

[0158] Then, the first trajectory is corrected according to the offset so that the aircraft continues to fly.

[0159] In another optional solution, the flight module 503 flies according to the generated first trajectory, specifically:

[0160] determining flight status data at each control moment during the flight process according to the first trajectory, the flight status data including at least one of position information, velocity information, and acceleration information;

[0161] At each control moment, the state of the aircraft is adjusted to the state indicated by the flight state data at each control moment.

[0162] It should be noted that the implementation of each module can also refer to Figure 3 The corresponding description of the method embodiment shown.

[0163] exist Figure 5In the described aircraft 50, during flight, the aircraft determines multiple flight moments based on at least one of flight speed, stick operation, and a preset distance threshold. Flight status data is then collected at these multiple determined flight moments, and a first trajectory is generated based on the flight status data. Finally, the aircraft flies along this first trajectory. Because the flight moments are determined based on at least one of flight speed, stick operation, and the preset distance threshold, the determined flight moments are generally critical moments in the aircraft's flight. Therefore, the first trajectory, determined based on the flight status data collected at these flight moments, can more accurately reconstruct the human-controlled flight trajectory.

[0164] See Figure 6 , Figure 6 An aircraft 60 provided by an embodiment of the present invention includes a processor 601 , a memory 602 , and a camera 603 . The processor 601 , the memory 602 , and the camera 603 are interconnected via a bus.

[0165] Memory 602 includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CD-ROM). Memory 602 is used for related instructions and data. Camera 603 is used to capture images during the flight of aircraft 70.

[0166] The processor 601 may be one or more central processing units (CPUs). In the case where the processor 601 is a CPU, the CPU may be a single-core CPU or a multi-core CPU.

[0167] The processor 601 in the aircraft 60 is configured to read the program code stored in the memory 602 and perform the following operations:

[0168] According to capturing first images and collecting flight status data at a plurality of flight moments, the flight status data includes at least one of position information, speed information, and acceleration information;

[0169] generating a first trajectory based on the flight status data collected at the plurality of flight moments;

[0170] capturing a second image during the flight along the first trajectory;

[0171] comparing the second image to the first image to determine an offset;

[0172] The first trajectory is corrected according to the offset so that the aircraft can continue to fly.

[0173] In yet another optional solution, the processor compares the second image with the first image to determine the offset, specifically by comparing the texture in the second image with the texture in the first image to determine the offset.

[0174] In yet another optional solution, after generating the first trajectory based on the flight status data collected at the plurality of flight moments, the processor is further configured to:

[0175] determining flight information at each control moment during the flight process according to the first trajectory, the flight status data including at least one of position information, velocity information, and acceleration information;

[0176] At each control moment, the state of the aircraft is adjusted to the state indicated by the flight state data at each control moment.

[0177] In another optional solution, the location information is determined based on at least one of GPS technology, visual angle positioning technology and inertial sensor positioning technology.

[0178] In yet another optional solution, before capturing the first image and collecting the flight status data at multiple flight moments, the processor is further configured to:

[0179] The plurality of flight moments are determined according to at least one of a flight speed, a stick operation, and a preset distance threshold during the flight of the aircraft.

[0180] In another optional solution, the processor determines the multiple flight moments according to at least one of a flight speed, a stick operation, and a preset distance threshold during the flight of the aircraft, specifically:

[0181] During the flight, each time a stick operation is detected for the aircraft is used as the flight time.

[0182] In another optional solution, the flight speed of the aircraft at each flight moment is not lower than a preset speed threshold.

[0183] In another optional solution, the distance between any two temporally adjacent flight moments of the aircraft is not less than a preset distance threshold.

[0184] It should be noted that the implementation of each operation can also refer to Figure 2 The corresponding description of the method embodiment shown.

[0185] exist Figure 6 In the described aircraft 60, during flight, the aircraft collects flight status data and captures a first image via a camera. A first trajectory is then generated using this flight status data. Subsequently, while flying along the first trajectory, a second image is captured in real time. The second image is then compared with the first image to determine the offset of the aircraft while flying along the first trajectory. This offset is then used to correct the first trajectory, and flight continues along the corrected first trajectory. Because the generation of the corrected first trajectory takes into account the aircraft's flight offset, it is possible to reduce iteration errors generated during flight, allowing the resulting first trajectory used for autonomous flight to closely replicate the human-controlled flight trajectory.

[0186] See Figure 7 , Figure 7 The embodiment of the present invention provides an aircraft 70, which includes a processor 701 and a memory 702. The processor 701 and the memory 702 can be interconnected via a bus. In addition, the aircraft can also include a camera 703 for taking pictures during the flight of the aircraft 70.

[0187] The memory 702 includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CD-ROM), and is used for related instructions and data.

[0188] The processor 701 may be one or more central processing units (CPUs). In the case where the processor 701 is a CPU, the CPU may be a single-core CPU or a multi-core CPU.

[0189] The processor 701 in the aircraft 70 is used to read the program code stored in the memory 702 and perform the following operations:

[0190] During the flight of the aircraft, a plurality of flight moments are determined based on at least one of a flight speed, a stick operation, and a preset distance threshold, and flight status data of the aircraft at the plurality of flight moments are collected, the flight status data including at least one of position information, speed information, and acceleration information;

[0191] generating a first trajectory according to the flight status data collected by the aircraft at the plurality of flight moments;

[0192] The aircraft is controlled to fly according to the generated first trajectory.

[0193] In an optional solution, the processor determines multiple flight moments during the flight of the aircraft based on at least one of flight speed, stick operation, and a preset distance threshold, and collects flight status data of the aircraft at the multiple flight moments, specifically:

[0194] During the flight, each time a stick operation is detected for the aircraft is used as the flight time, and the flight status data of the aircraft at that flight time is collected.

[0195] In another optional solution, the flight speed of the aircraft at each flight moment is not lower than a preset speed threshold.

[0196] In another optional solution, the distance between any two temporally adjacent flight moments of the aircraft is not less than a preset distance threshold.

[0197] In another optional solution, the location information is determined based on at least one of GPS technology, visual angle positioning technology and inertial sensor positioning technology.

[0198] In yet another optional solution, the processor determines multiple flight moments during the flight of the aircraft based on at least one of flight speed, stick operation, and a preset distance threshold, and collects flight status data of the aircraft at the multiple flight moments, specifically:

[0199] During the flight of the aircraft, a plurality of flight moments are determined based on at least one of a flight speed, a stick operation, and a preset distance threshold, and a first image is captured and flight status data of the aircraft is collected at the plurality of flight moments.

[0200] The processor flies according to the generated first trajectory, specifically:

[0201] capturing a second image while flying along the first trajectory;

[0202] comparing the second image to the first image to determine an offset;

[0203] The first trajectory is corrected according to the offset so that the aircraft continues to fly.

[0204] In another optional solution, the processor flies according to the generated first trajectory, specifically:

[0205] determining flight status data at each control moment during the flight process according to the first trajectory, the flight status data including at least one of position information, velocity information, and acceleration information;

[0206] At each control moment, the state of the aircraft is adjusted to the state indicated by the flight state data at each control moment.

[0207] It should be noted that the implementation of each operation can also refer to Figure 3 The corresponding description of the method embodiment shown.

[0208] exist Figure 7 In the described aircraft 70, during flight, the aircraft determines multiple flight moments based on at least one of flight speed, stick manipulation, and a preset distance threshold. Flight status data is then collected at these multiple determined flight moments, and a first trajectory is generated based on the flight status data. Finally, the aircraft flies along this first trajectory. Because the flight moments are determined based on at least one of flight speed, stick manipulation, and the preset distance threshold, the determined flight moments are generally critical moments in the aircraft's flight. Therefore, the first trajectory, determined based on the flight status data collected at these flight moments, can more accurately reconstruct the human-controlled flight trajectory.

[0209] An embodiment of the present invention further provides a chip system, which includes at least one processor, a memory and an interface circuit, wherein the memory, the transceiver and the at least one processor are interconnected via a line, and the at least one memory stores program instructions; when the program instructions are executed by the processor, Figure 2 or Figure 3 The method flow shown.

[0210] The embodiment of the present invention further provides a computer-readable storage medium having instructions stored therein, which, when executed on a network device, implements Figure 2 or Figure 3 The method flow shown.

[0211] The embodiment of the present invention further provides a computer program product, which, when executed on a processor, implements Figure 2 or Figure 3 The method flow shown.

[0212] In summary, in an embodiment of the present invention, the aircraft collects flight status data and captures a first image via a camera during flight. A first trajectory is then generated using this flight status data. Subsequently, a second image is captured in real time while the aircraft is flying along the first trajectory. The second image is then compared with the first image to determine the offset of the aircraft while flying along the first trajectory. This offset is then used to correct the first trajectory, and flight continues along the corrected first trajectory. Because the generation of the corrected first trajectory takes into account the offset of the aircraft during flight, it is possible to reduce iterative errors generated during flight, ensuring that the final first trajectory generated for autonomous flight closely replicates the human-controlled flight trajectory.

[0213] It can be understood that the above disclosure is only a part of the embodiments of the present invention, and of course it cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiments are implemented, and equivalent changes made in accordance with the claims of the present invention are still within the scope of the invention.

Claims

1. A flight trajectory replay method, characterized in that: include: During flight, an aircraft determines multiple flight times based on at least one of flight speed, stick operation, and a preset distance threshold. The method for determining the multiple flight times based on the stick operation during flight includes: determining each time a stick operation is detected for the aircraft during flight as a flight time; the method for determining the multiple flight times based on flight speed during flight includes: ensuring that the flight speed at each flight time is no less than a preset speed threshold; and the method for determining the multiple flight times based on the preset distance threshold during flight includes: ensuring that the distance between any two temporally adjacent flight times is no less than the preset distance threshold. The aircraft captures first images and collects flight status data at the plurality of flight moments, the flight status data including at least one of position information, speed information, and acceleration information; The aircraft generates a first trajectory based on the flight status data collected at the multiple flight moments, wherein the smoothness of the first trajectory is higher than the smoothness of the trajectory of the aircraft at the multiple flight moments; The aircraft flies according to the generated first trajectory.

2. The method according to claim 1, characterized in that After the aircraft generates a first trajectory according to the flight status data collected at the plurality of flight moments, the method further includes: The aircraft determines flight status data at each control moment during the flight process according to the first trajectory, the flight status data including at least one of position information, velocity information, and acceleration information; The aircraft adjusts its own state to the state indicated by the flight state data at each control moment.

3. The method according to claim 1, characterized in that Also includes: The aircraft captures a second image while flying along the first trajectory; The aircraft compares the second image with the first image to determine an offset; The aircraft corrects the first trajectory according to the offset so that the aircraft continues to fly.

4. The method according to claim 1, wherein The first trajectory is substantially the same as the trajectory of the aircraft at the multiple flight moments.

5. The method according to claim 1, wherein The first image is associated with the position information.

6. The method according to claim 1 or 5, characterized in that The location information is determined based on at least one of GPS technology, visual angle positioning technology, and inertial sensor positioning technology.

7. An aircraft, characterized in that: The aircraft includes a processor and a memory, wherein the memory is used to store program instructions, and the processor is used to call the program instructions to perform the following operations: During flight, multiple flight times are determined based on at least one of flight speed, stick operation, and a preset distance threshold. The method for determining the multiple flight times based on the stick operation during flight includes: taking each time a stick operation is detected for the aircraft during flight as a flight time; the method for determining the multiple flight times based on flight speed during flight includes: the flight speed at each flight time is not less than a preset speed threshold; the method for determining the multiple flight times based on the preset distance threshold during flight includes: the distance between any two temporally adjacent flight times is not less than the preset distance threshold. capturing first images and collecting flight status data at the plurality of flight moments, the flight status data including at least one of position information, speed information, and acceleration information; generating a first trajectory based on the flight status data collected at the plurality of flight moments, wherein the first trajectory has a smoothness higher than a smoothness of the trajectory of the aircraft at the plurality of flight moments; The aircraft is controlled to fly according to the generated first trajectory.

8. The aircraft according to claim 7, characterized in that After generating the first trajectory according to the flight status data collected at the plurality of flight moments, the processor is further configured to: determining flight information at each control moment during the flight process according to the first trajectory, wherein the flight status data includes at least one of position information, speed information, and acceleration information; At each control moment, the state of the aircraft is adjusted to the state indicated by the flight state data at each control moment.

9. The aircraft according to claim 7, characterized in that The processor is further configured to: capturing a second image while flying along the first trajectory; comparing the second image to the first image to determine an offset; The first trajectory is corrected according to the offset so that the aircraft continues to fly.

10. The aircraft according to claim 7, characterized in that The first trajectory is substantially the same as the trajectory of the aircraft at the multiple flight moments.

11. The aircraft according to claim 7, characterized in that The first image is associated with the position information.

12. The aircraft according to claim 7 or 11, characterized in that The location information is determined based on at least one of GPS technology, visual angle positioning technology, and inertial sensor positioning technology.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores program instructions, and when the program instructions are executed on a processor, the method according to any one of claims 1 to 6 is implemented.

14. A computer program product, characterized in that When the computer program product is run on a processor, the method according to any one of claims 1 to 6 is implemented.

Images