An aerial photography tracking zoom method and system

Abstract

The present disclosure relates to the field of UAV image processing technology, and specifically relates to an aerial photography tracking zoom method and system. The method includes: calculating the fixed acceleration of the UAV according to the initial shooting distance, specified frame rate, specified frame zoom ratio, and progress determination threshold between the acquired shooting device and the shooting object, where the shooting device is arranged on the UAV; while shooting a video at the specified frame rate and controlling the shooting device to perform digital zoom on the shooting video according to the specified frame zoom ratio, starting from the initial shooting distance, controlling the UAV to fly away from the shooting object along the optical axis direction of the shooting device with uniform acceleration according to the fixed acceleration, so as to achieve aerial photography tracking zoom. The present disclosure simplifies the flight control difficulty of the UAV, can achieve more flexible digital zoom and matching flight speed control, thereby realizing the automated management of tracking zoom, improving the shooting quality of the video, and can be widely used in multiple fields such as movie shooting and advertisement production.

Description

Technical Field

[0001] The present disclosure relates to the technical field of UAV image processing, and particularly to an aerial photography dolly zoom method and system. Background Art

[0002] The Hitchcock zoom, also known as the slider zoom or dolly zoom, is a filming method in which, during video shooting, the camera (or video camera) is moved backward or forward while zooming in or out, so as to keep the size of the subject in the frame unchanged while the size of the background changes significantly, and it has a strong visual impact.

[0003] In traditional video shooting, the implementation of the Hitchcock zoom relies on manual operation: the photographer needs to manually control the optical zoom of the lens and move the camera with the help of a dolly. This process is not only complex to operate, but also has high requirements for equipment and technology.

[0004] In recent years, with the rapid development of UAV technology, some UAV-based solutions have emerged, which simplify the implementation process of the Hitchcock zoom by autonomously moving the UAV to control the optical zoom of the camera. However, these solutions still have certain limitations. For example, existing UAVs usually need to carry a camera equipped with an optical zoom lens, and the limited load space and strict hardware integration requirements of UAVs further raise the standards for the performance of optical zoom lenses, resulting in a significant increase in economic costs. In addition, existing solutions often only support a limited number of zoom positions and lack the ability to automatically control the Hitchcock zoom, thus limiting their flexibility and practicality. Summary of the Invention

[0005] To solve the problems in the related art, embodiments of the present disclosure provide an aerial photography dolly zoom method and system.

[0006] In a first aspect, an aerial photography dolly zoom method provided in embodiments of the present disclosure includes:

[0007] Obtaining an initial shooting distance, a specified frame rate, a specified frame zoom ratio, and a progress determination threshold between a shooting device and a shooting object, where the shooting device is arranged on a UAV, and the progress determination threshold includes any one of the following: maximum aerial photography zoom time, maximum zoom ratio, and maximum flight speed;

[0008] Calculating a fixed acceleration of the UAV according to the initial shooting distance, the progress determination threshold, the specified frame rate, and the specified frame zoom ratio;

[0009] While shooting a video at the specified frame rate and digitally zooming the captured video by the shooting device according to the specified frame zoom ratio, the drone is controlled to fly away from the subject along the optical axis direction of the shooting device with a uniform acceleration starting from the initial shooting distance according to the fixed acceleration, so as to achieve aerial shooting tracking zooming.

[0010] According to an embodiment of the present disclosure, it further includes:

[0011] During the aerial shooting tracking zooming process, a progress determination parameter corresponding to the progress determination threshold is obtained, and the progress determination parameter includes any one of the aerial shooting zooming time, the zoom ratio, and the flight speed;

[0012] When the progress determination parameter reaches or exceeds the corresponding progress determination threshold, the aerial shooting tracking zooming process ends.

[0013] According to an embodiment of the present disclosure:

[0014] The maximum aerial shooting zooming time is the minimum value between the first maximum aerial shooting zooming time and the second maximum aerial shooting zooming time, where the first maximum aerial shooting zooming time is related to the maximum zoom ratio, and the second maximum aerial shooting zooming time is related to the maximum flight speed.

[0015] According to an embodiment of the present disclosure:

[0016] The first maximum aerial shooting zooming time and the maximum zoom ratio satisfy the following formula:

[0017] ;

[0018] The second maximum aerial shooting zooming time and the maximum flight speed satisfy the following formula:

[0019] ;

[0020] The maximum aerial shooting zooming time and the first maximum aerial shooting zooming time and the second maximum aerial shooting zooming time satisfy the following formula:

[0021] ;

[0022] where d is the initial shooting distance, is the specified frame rate, and k is the specified frame zoom ratio.

[0023] According to an embodiment of the present disclosure, when the progress determination parameter reaches or exceeds the corresponding progress determination threshold, ending the aerial photography tracking zoom process includes:

[0024] When the aerial photography zoom time t reaches or exceeds the maximum aerial photography zoom time the aerial photography tracking zoom process is ended.

[0025] According to an embodiment of the present disclosure, the method further includes:

[0026] When digitally zooming the captured video by the imaging device according to the specified frame zoom ratio, each frame of the captured video is enlarged by at least 2 pixels frame by frame;

[0027] The zoom ratio is related to the specified frame zoom ratio.

[0028] According to an embodiment of the present disclosure:

[0029] When each frame of the captured video is enlarged by at least 2 pixels frame by frame, the specified frame zoom ratio k satisfies the following formula:

[0030] ;

[0031] The relationship between the zoom ratio n and the specified frame zoom ratio k satisfies the following formula:

[0032] , ;

[0033] where t is the aerial photography zoom time, is the large aerial photography zoom time, is the maximum flight speed, w is the width of the first frame of the captured video, h is the height of the first frame of the captured video, and the first frame is the frame captured by the imaging device at the initial shooting distance.

[0034] According to an embodiment of the present disclosure, when the progress determination parameter reaches or exceeds the corresponding progress determination threshold, ending the aerial photography tracking zoom process includes:

[0035] When the zoom ratio reaches or exceeds the maximum zoom ratio, the aerial photography tracking zoom process is ended.

[0036] According to an embodiment of the present disclosure, calculating the fixed acceleration of the unmanned aerial vehicle according to the initial shooting distance, the progress determination threshold, the specified frame rate, and the specified frame zoom ratio includes calculating the fixed acceleration a according to the following formula:

[0037] ;

[0038] where d is the initial shooting distance, fps is the specified frame rate, and k is the specified frame zoom ratio, is the maximum aerial photography zoom time.

[0039] According to an embodiment of the present disclosure, controlling the drone to fly away from the photographed object at a uniform acceleration along the optical axis direction of the photographing device according to the fixed acceleration includes: the drone flying at the following flight speed to achieve the uniform acceleration flight:

[0040] = ;

[0041] ;

[0042] where is the initial flight speed and t is the aerial photography zoom time.

[0043] According to an embodiment of the present disclosure, when the progress determination parameter reaches or exceeds the corresponding progress determination threshold, ending the aerial photography tracking zoom process includes:

[0044] When the flight speed reaches or exceeds the maximum flight speed, end the aerial photography tracking zoom process.

[0045] In a second aspect, an aerial photography tracking zoom system is provided in an embodiment of the present disclosure, including a main control unit, a flight control module, and a digital zoom module; the flight control module is arranged on the drone, the digital zoom module is arranged on the photographing device, and the main control unit is communicatively connected to the flight control module and the digital zoom module;

[0046] The main control unit is configured to:

[0047] Obtain the initial shooting distance, the specified frame rate, the specified frame zoom ratio, and the progress determination threshold between the photographing device and the photographed object, the photographing device is arranged on the drone, and the progress determination threshold includes any one of the following: the maximum aerial photography zoom time, the maximum zoom multiple, and the maximum flight speed;

[0048] Calculate the fixed acceleration of the drone according to the initial shooting distance, the progress determination threshold, the specified frame rate, and the specified frame zoom ratio;

[0049] While controlling the shooting device to shoot a video at the specified frame rate and controlling the digital zoom module to perform digital zoom on the shot video according to the specified frame zoom ratio, the flight control module is controlled to control the drone to fly away from the shooting object along the optical axis direction of the shooting device with a uniform acceleration starting from the initial shooting distance according to the fixed acceleration, so as to achieve aerial shooting dolly zoom.

[0050] According to an embodiment of the present disclosure, the main control unit includes a progress detection unit;

[0051] The progress detection unit is configured to obtain a progress determination parameter corresponding to the progress determination threshold during the aerial shooting dolly zoom process, and the progress determination parameter includes any one of the aerial shooting zoom time, zoom ratio, and flight speed;

[0052] The main control unit is further configured to end the aerial shooting dolly zoom process when the progress determination parameter reaches or exceeds the corresponding progress determination threshold.

[0053] According to an embodiment of the present disclosure, the system further includes an encoding and output module, and the encoding and output module is disposed on the shooting device;

[0054] The encoding and output module is configured to encode and output a dolly zoom video after completing the aerial shooting dolly zoom.

[0055] In a third aspect, an embodiment of the present disclosure provides a computer-readable storage medium, on which computer instructions are stored, and when the computer instructions are executed by a processor, the method described in any item of the first aspect is implemented.

[0056] In a fourth aspect, an embodiment of the present disclosure provides a computer program product, including computer instructions, and when the computer instructions are executed by a processor, the method described in any item of the first aspect is implemented.

[0057] According to the technical solution provided by the embodiment of the present disclosure, the fixed acceleration of the drone is calculated according to the obtained initial shooting distance, specified frame rate, specified frame zoom ratio, and progress determination threshold between the shooting device and the shooting object. The shooting device is disposed on the drone, and the progress determination threshold includes any one of the following: maximum aerial shooting zoom time, maximum zoom ratio, and maximum flight speed; while shooting a video at the specified frame rate and controlling the shooting device to perform digital zoom on the shot video according to the specified frame zoom ratio, the drone is controlled to fly away from the shooting object along the optical axis direction of the shooting device with a uniform acceleration starting from the initial shooting distance, so as to achieve aerial shooting dolly zoom.

[0058] The present disclosure reduces the reliance on high-cost optical zoom lenses, thereby significantly reducing the hardware cost. At the same time, it simplifies the flight control requirements of the drone and can dynamically adjust the flight speed and zoom speed according to user needs. It can achieve more flexible digital zoom and matching flight speed control, thus realizing the automated management of dolly zoom, reducing the need for manual intervention, and improving the shooting efficiency and consistency. It is particularly suitable for application scenarios that require repeated or precise control, such as movie shooting, advertising production, live sports events, and many other fields. Through automated management and simplified flight control, the operation difficulty is reduced, enabling even non-professionals to easily achieve high-quality dolly zoom shooting.

[0059] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and do not limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In conjunction with the drawings, through the following detailed description of non-limiting embodiments, other features, objects, and advantages of the present disclosure will become more apparent. In the drawings:

[0061] Figure 1 A flowchart showing a method for aerial dolly zoom according to an embodiment of the present disclosure;

[0062] Figure 2 A schematic diagram showing the digital zoom process in a method for aerial dolly zoom according to an embodiment of the present disclosure;

[0063] Figure 3 A schematic diagram showing the relationship between the shooting object and the displayed image in a method for aerial dolly zoom according to an embodiment of the present disclosure;

[0064] Figure 4 A schematic diagram showing a comparison between a linear flight speed function and a non-linear flight speed function in an embodiment of the present disclosure;

[0065] Figure 5 A flowchart showing aerial dolly zoom using a method for aerial dolly zoom according to an embodiment of the present disclosure;

[0066] Figure 6 A block diagram showing the structure of an aerial dolly zoom system according to an embodiment of the present disclosure;

[0067] Figure 7 A schematic diagram showing the structure of a computer system suitable for implementing the method according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0068] In the following, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. In addition, for clarity, parts unrelated to the description of the exemplary embodiments are omitted in the drawings.

[0069] In the present disclosure, it should be understood that terms such as "including" or "having" are intended to indicate the existence of features, numbers, steps, actions, components, parts, or combinations thereof disclosed in this specification, and do not preclude the possibility of the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0070] It should also be noted that, without conflict, the embodiments in the present disclosure and the features in the embodiments can be combined with each other. The present disclosure will be described in detail below with reference to the drawings and in combination with the embodiments.

[0071] In the present disclosure, when it comes to operations of obtaining user information or user data or presenting user information or user data to others, such operations are all operations authorized, confirmed by the user, or actively selected by the user.

[0072] As mentioned above, the prior art has achieved the control of the optical zoom of the camera through the autonomous movement of the drone, thus simplifying the implementation process of the Hitchcock zoom. However, these solutions still have significant limitations. First, existing drones usually need to carry a camera equipped with an optical zoom lens, and the limited load space and strict hardware integration requirements of the drone further raise the standard for the performance of the optical zoom lens, resulting in a substantial increase in economic costs. Second, existing solutions often only support a limited number of zoom positions and lack the ability to automatically control the Hitchcock zoom, which severely limits their flexibility and practicality. Therefore, there is an urgent need for a more efficient, economical, and flexible solution to overcome these challenges.

[0073] According to the technical solution of the present disclosure, by obtaining the initial shooting distance, specified frame rate, specified frame scaling ratio, and progress determination threshold between the shooting device and the shooting object, the shooting device is arranged on the drone, and the progress determination threshold includes any one of the following: maximum aerial photography zoom time, maximum zoom ratio, maximum flight speed; calculating the fixed acceleration of the drone according to the initial shooting distance, the progress determination threshold, the specified frame rate, and the specified frame scaling ratio; while shooting a video at the specified frame rate and controlling the shooting device to perform digital zoom on the shooting video according to the specified frame scaling ratio, controlling the drone to fly away from the shooting object along the optical axis direction of the shooting device with uniform acceleration starting from the initial shooting distance, so as to achieve aerial tracking zoom.

[0074] The present disclosure simplifies the flight control requirements of the drone and can dynamically adjust the flight speed and zoom speed according to user needs. It can achieve more flexible digital zoom and matching flight speed control, thereby realizing the automated management of tracking zoom and reducing the need for manual intervention. In addition, through automated management and simplified flight control, the operation difficulty is reduced, enabling even non-professionals to easily achieve high-quality tracking zoom shooting.

[0075] Figure 1 The flowchart of an aerial tracking zoom method according to an embodiment of the present disclosure is shown.

[0076] As Figure 1 shown, the method includes the following steps S101 to S103:

[0077] In step S101, obtain the initial shooting distance, specified frame rate, specified frame zoom ratio, and progress determination threshold between the shooting device and the shooting object.

[0078] According to an embodiment of the present disclosure, the shooting device is provided on the drone. The shooting device is any device that can shoot videos, such as a camera, a video camera, a smart phone, a camera, and so on. The shooting device can be fixed on the drone or detachably connected to the drone, and the present disclosure does not limit this.

[0079] Among them, the above parameters (initial shooting distance, specified frame rate, specified frame zoom ratio, progress determination threshold) are obtained through user input.

[0080] According to an embodiment of the present disclosure, the progress determination threshold includes any one of the following: the maximum aerial zoom time, the maximum zoom multiple, and the maximum flight speed.

[0081] In the present disclosure, the initial shooting distance is the distance between the shooting device and the shooting object at the initial position, and the initial position is the position where the shooting device and the shooting object are in a relatively static state. It can be set as needed. Or it can be measured using sensors in the actual scene, such as laser ranging, ultrasonic, etc. It can also be calculated using machine learning algorithms, for example, estimated through an AI network.

[0082] The specified frame rate is the number of frames per second shot by the shooting device. The frame rate of the shooting device is closely related to its hardware performance (sensor, processor, memory) and software functions (frame rate setting, shooting mode), and can be selected according to actual needs (such as resolution, frame rate, shooting scene).

[0083] The specified frame zoom ratio refers to the ratio of image enlargement achieved by adjusting the picture size during the processing of the captured video. It is used to describe the degree of change of the picture from the original size to the target size.

[0084] According to an embodiment of the present disclosure, when controlling the imaging device to perform digital zoom on the captured video according to the specified frame zoom ratio, each image frame of the captured video is enlarged by at least 2 pixels frame by frame. That is, each frame of the image is enlarged by at least two pixels to ensure that there is at least one pixel indentation on each side of the captured image frame.

[0085] For example, at the initial shooting distance, the imaging device is used to capture a first image frame of a shooting object, and the size of the first image frame includes the number of horizontal pixels w and the number of vertical pixels h. The first image frame is enlarged according to the execution frame zoom ratio k to obtain a second image frame adjacent to the first image frame. Then the size of the second image frame is: the number of horizontal pixels wk and the number of vertical pixels hk. Therefore, the specified frame zoom ratio k needs to satisfy the following conditions: , so it can be inferred that , that is, when each image frame of the captured video is enlarged by at least 2 pixels frame by frame, the specified frame zoom ratio k satisfies the following formula: ; in other words, the specified frame zoom ratio is greater than or equal to the larger value of the value obtained by dividing 2 by the width of the first image frame plus 1 and the value obtained by dividing 2 by the height of the first image frame plus 1.

[0086] In step S102, the fixed acceleration of the drone is calculated according to the initial shooting distance, the progress determination threshold, the specified frame rate, and the specified frame zoom ratio.

[0087] In step S103, while shooting a video at the specified frame rate and controlling the imaging device to perform digital zoom on the captured video according to the specified frame zoom ratio, the drone is controlled to fly away from the shooting object along the optical axis direction of the imaging device with a uniform acceleration starting from the initial shooting distance, so as to achieve aerial photography push-rail zoom.

[0088] Wherein, when controlling the imaging device to perform digital zoom on the captured video according to the specified frame zoom ratio, if the size (number of horizontal pixels and number of vertical pixels) of the enlarged image frame is a decimal, a rounding operation (for example, rounding) needs to be performed.

[0089] In the present disclosure, digital zooming of a captured video includes: image magnification and image cropping. Among them, the order of image magnification and cropping can be swapped, which does not affect the result of the finally obtained digital zooming. Specifically, image magnification is to magnify the pixels of each frame image of the captured video by using the specified frame zoom ratio, and image cropping is to re-crop the size of each magnified frame image to the size of the original image (i.e., the size before magnification). The digital zooming of the captured video is achieved through the image magnification and image cropping.

[0090] Figure 2 Fig. shows a schematic diagram of the image zooming process in an aerial photography tracking zooming method according to an embodiment of the present disclosure.

[0091] As Figure 2 shown, when re-cropping the magnified image to the size of the original image, the position of the cropping frame is the central rectangular area of the magnified image. Assume that the size of the original image is , and the size of the magnified image is . Taking the upper left vertex coordinate of the magnified image as the origin, the upper left vertex coordinate of the cropped image is: , and the size is .

[0092] By adopting digital zooming instead of optical zooming, the present disclosure reduces the dependence on high-cost optical zoom lenses, thus significantly reducing the hardware cost. At the same time, the introduction of digital zooming makes the zooming process more flexible, enabling a wider zoom range and more precise zoom control.

[0093] According to an embodiment of the present disclosure, during the aerial photography tracking zooming process, a progress determination parameter corresponding to the progress determination threshold is obtained. The progress determination parameter includes any one of the aerial photography zooming time, zoom ratio, and flight speed; when the progress determination parameter reaches or exceeds the corresponding progress determination threshold, the aerial photography tracking zooming process ends.

[0094] Therefore, by combining digital zooming and unmanned aerial vehicle flight control, the present disclosure realizes the automated management of tracking zooming. This not only reduces the need for manual intervention but also improves the shooting efficiency and consistency, and is particularly suitable for application scenarios that require repeated or precise control.

[0095] The present disclosure can also dynamically adjust the flight speed and zoom speed according to actual needs, so that the speed of flight control can better match the speed of digital zooming, reducing the picture jitter or instability phenomenon caused by the mismatch between the flight speed and the zoom speed, thereby improving the shooting quality. And through automated management and simplified flight control, the operation difficulty is reduced, enabling even non-professionals to easily achieve high-quality tracking zooming shooting.

[0096] Further, the maximum aerial zoom time is the minimum value between the first maximum aerial zoom time and the second maximum aerial zoom time, where the first maximum aerial zoom time is related to the maximum zoom ratio, and the second maximum aerial zoom time is related to the maximum flight speed.

[0097] Specifically, the first maximum aerial zoom time and the maximum zoom ratio satisfy the following formula:

[0098] ;

[0099] The second maximum aerial zoom time and the maximum flight speed satisfy the following formula:

[0100] ;

[0101] The maximum aerial zoom time and the first maximum aerial zoom time and the second maximum aerial zoom time satisfy the following formula:

[0102] ;

[0103] where d is the initial shooting distance, is the specified frame rate, and k is the specified frame zoom ratio.

[0104] Further, the zoom ratio is related to the specified frame zoom ratio.

[0105] where the relationship between the zoom ratio n and the specified frame zoom ratio k satisfies the following formula:

[0106] , , ;

[0107] where t is the aerial zoom time, is the large aerial zoom time, is the maximum flight speed, w is the width of the first image frame of the captured video, h is the height of the first image frame of the captured video, and the first image frame is the image frame captured by the capturing device at the initial shooting distance.

[0108] According to an embodiment of the present disclosure, when the zoom ratio reaches or exceeds the maximum zoom ratio, the aerial tracking zoom process ends.

[0109] According to an embodiment of the present disclosure, calculating the fixed acceleration of the drone according to the initial shooting distance, the progress determination threshold, the specified frame rate, and the specified frame magnification factor includes calculating the fixed acceleration a according to the following formula:

[0110] ;

[0111] where d is the initial shooting distance, fps is the specified frame rate, k is the specified frame magnification factor, is the maximum aerial photography zoom time.

[0112] Controlling the drone to fly away from the shooting object at a uniform acceleration along the optical axis direction of the shooting device according to the fixed acceleration includes: the drone flying at a uniform acceleration according to the following flight speed to achieve the uniform acceleration flight:

[0113] = ;

[0114] ;

[0115] where, is the initial flight speed and t is the aerial photography zoom time.

[0116] According to an embodiment of the present disclosure, when the aerial photography zoom time t reaches or exceeds the maximum aerial photography zoom time , the aerial photography dolly zoom process ends.

[0117] The following details the calculation of the fixed acceleration of the drone according to the initial shooting distance, the progress determination threshold, the specified frame rate, and the specified frame magnification factor.

[0118] Figure 3 FIG. shows a schematic diagram of the relationship between the shooting object and the displayed image in an aerial photography dolly zoom method according to an embodiment of the present disclosure. Among them, the displayed image is the image displayed after the shooting device shoots the shooting object.

[0119] As Figure 3 shown, assume that the actual size of the shooting object is h 实 , and its height in the shooting device (such as a camera) is h 显 when the shooting distance is d and the focal length is set to f. Then, in order to achieve dolly zoom, that is, when the shooting distance is d + Δd and the focal length is set to f', that is, when the shooting device moves a specified distance Δd, it is required that the height of the shooting object in the shooting device still remains h 显 , then according to the principle of similar triangles, it can be obtained that:

[0120] ;

[0121] ;

[0122] Thus, the corresponding relationship between the focal length and the shooting distance can be deduced as:

[0123] ;

[0124] It is known that according to the definition of the digital zoom ratio, it can be known that: , then according to the corresponding relationship between the focal length and the shooting distance, it can be obtained that:

[0125] ;

[0126] At the same time, the corresponding relationship between the digital zoom ratio n and the specified frame scaling ratio k, the aerial photography zoom time t, and the specified frame rate fps satisfies the following formula:

[0127] ;

[0128] Then the corresponding relationship between the initial shooting distance d, the specified distance and the specified frame scaling ratio k, the specified frame rate fps, and the aerial photography zoom time t satisfies the following formula:

[0129] .

[0130] Furthermore, by taking the derivative of the above formula with respect to time t, the flight speed function of the drone can be obtained:

[0131] ;

[0132] That is, when the shooting device performs digital zoom with a scaling ratio of k per frame, the flight speed of the drone along the optical axis of the shooting device is .

[0133] Taking the derivative of the flight speed function with respect to time t again, the acceleration function of the drone can be obtained:

[0134] .

[0135] The inventor noticed that if according to the above flight speed function or the acceleration function It is actually difficult to perform flight control on an unmanned aerial vehicle (UAV). Since a UAV is a complex system, its flight control depends on the coordinated operation of multiple subsystems, such as the power system, the control surface control system, the sensor network, and the flight control system, etc. These subsystems interact with each other and jointly determine the flight state of the UAV. Therefore, nonlinear control of the UAV requires comprehensive consideration of multiple factors, including the dynamic characteristics of the UAV, external environmental interference, sensor accuracy, etc., which makes the design and implementation of the control strategy overly complex.

[0136] Due to the limitations of the resolution of the shooting device or the constraints of the UAV's maneuverability, the maximum zoom duration during aerial photography is relatively limited. In this case, by reasonably approximating the acceleration function and controlling the flight of the UAV based on the approximated acceleration value, the execution process of flight control can be significantly simplified, thereby making flight control more practical and greatly reducing its complexity.

[0137] Therefore, by analyzing the above acceleration function it can be seen that since the values of the specified frame zoom ratio k and the maximum aerial photography zoom time are generally small, the value of ln(k) is also very small, and the values of d and fps have little impact on the final value. Therefore, the acceleration when t is the median value of the maximum aerial photography zoom time can be taken as the fixed acceleration of the UAV , thereby realizing a linear approximation of the nonlinear acceleration function, and the following can be obtained: .

[0138] Furthermore, the UAV realizes uniformly accelerated flight according to the following flight speed :

[0139] = ;

[0140] .

[0141] Next, a specific embodiment is used to illustrate the principle of taking the acceleration when t is the median value of the maximum aerial photography zoom time as the fixed acceleration of the UAV . Those skilled in the art should understand that the specific values of the specified frame zoom ratio k, the maximum aerial photography zoom time , the initial shooting distance d, and the frame rate fps are only for illustrative purposes and do not serve as technical means for limiting the protection scope of the present disclosure.

[0142] Assume that at the initial shooting distance, the image size of the captured video is: 1920*1080, and each zoom scales 4 pixels, then:

[0143] ;

[0144] Set = 1.0037, d = 10, fps = 30, = 10.

[0145] Substitute the above four parameters into the formula of the flight speed function of the drone to obtain the non - linear flight speed function .

[0146] Substitute the above four parameters into the flight speed formula: to obtain the linear flight speed function .

[0147] Figure 4 Fig. shows a schematic diagram for comparing the linear flight speed function and the non - linear flight speed function in the embodiments of the present disclosure.

[0148] As Figure 4 shown, since k≈1, the non - linear flight speed function has a gentle curve, and ln(k) is very small. The d and fps coefficients have little influence on the final value. Generally, the maximum aerial photography zoom time is small, so it can be considered that the linear flight speed function is approximately the same as the non - linear flight speed function .

[0149] In the present disclosure, by using the flight control of the drone and the digital zoom of the shooting device, the enlarged zoom in the aerial photography dolly zoom is realized. The present disclosure can also perform an inverse transformation on the enlarged zoom video when processing the dolly zoom video subsequently, so as to realize the reduced zoom of the shooting video, enriching the shooting mode of the present disclosure, and making the video shot by using the aerial photography dolly zoom of the present disclosure not limited to the effect of enlarged zoom only.

[0150] Figure 5 Fig. shows a flowchart of performing aerial photography dolly zoom according to an aerial photography dolly zoom method in the embodiments of the present disclosure.

[0151] As Figure 5 shown, the aerial photography dolly zoom method can be applied in the main control unit, for example, in the image processing module (CPU) of the shooting device, or in a control unit independent of the shooting device and the drone.

[0152] First, the user inputs parameters (the initial shooting distance between the shooting device and the shooting object, the specified frame rate, the specified frame scaling ratio, the progress determination threshold).

[0153] Then, the main control unit calculates the fixed acceleration of the drone according to the initial shooting distance, the progress determination threshold, the specified frame rate, and the specified frame zoom ratio.

[0154] Finally, while the main control unit shoots a video at the specified frame rate and controls the shooting device to perform digital zoom on the captured video according to the specified frame zoom ratio, it controls the drone to fly away from the shooting object along the optical axis direction of the shooting device with a uniform acceleration starting from the initial shooting distance according to the fixed acceleration.

[0155] Among them, during the aerial photography dolly zoom process, progress detection is required, that is, obtaining a progress determination parameter corresponding to the progress determination threshold, and the progress determination parameter includes any one of the aerial photography zoom time, the zoom ratio, and the flight speed; and when the progress determination parameter reaches or exceeds the corresponding progress determination threshold, the aerial photography dolly zoom process ends. For example, when the aerial photography zoom time reaches or exceeds the maximum aerial photography zoom time, or when the zoom ratio reaches or exceeds the maximum zoom ratio, or when the flight speed reaches or exceeds the maximum flight speed, the aerial photography dolly zoom process ends.

[0156] If the progress determination parameter does not reach the corresponding progress determination threshold, then continue with digital zoom and flight control of the drone until the progress determination parameter reaches or exceeds the corresponding progress determination threshold.

[0157] Figure 6 The structural block diagram of an aerial photography dolly zoom system according to an embodiment of the present disclosure is shown.

[0158] As Figure 6 shown, the aerial photography dolly zoom system includes: a main control unit, a flight control module, and a digital zoom module; the flight control module is arranged on the drone, the digital zoom module is arranged on the shooting device, and the main control unit is communicatively connected to the flight control module and the digital zoom module.

[0159] The main control unit is configured to:

[0160] Obtain the initial shooting distance, the specified frame rate, the specified frame zoom ratio, and the progress determination threshold between the shooting device and the shooting object, the shooting device is arranged on the drone, and the progress determination threshold includes any one of the following: the maximum aerial photography zoom time, the maximum zoom ratio, and the maximum flight speed;

[0161] Calculate the fixed acceleration of the drone according to the initial shooting distance, the progress determination threshold, the specified frame rate, and the specified frame zoom ratio;

[0162] While shooting a video at the specified frame rate by controlling the shooting device and digitally zooming the captured video by controlling the digital zoom module according to the specified frame zoom ratio, the flight control module is controlled to fly the drone away from the shooting object along the optical axis direction of the shooting device with a uniform acceleration starting from the initial shooting distance according to the fixed acceleration, so as to realize aerial push-rail zooming.

[0163] According to an embodiment of the present disclosure, the main control unit includes a progress detection unit;

[0164] The progress detection unit is configured to obtain a progress determination parameter corresponding to the progress determination threshold during the aerial push-rail zooming process, and the progress determination parameter includes any one of the aerial zooming time, the zoom ratio, and the flight speed;

[0165] The main control unit is further configured to end the aerial push-rail zooming process when the progress determination parameter reaches or exceeds the corresponding progress determination threshold.

[0166] According to an embodiment of the present disclosure, the system further includes an encoding and output module, and the encoding and output module is disposed on the shooting device;

[0167] The encoding and output module is configured to encode and output a push-rail zoom video after completing the aerial push-rail zooming.

[0168] The present disclosure simplifies the flight control requirements of the drone, further reduces the system complexity and implementation cost, and can be widely applied to multiple fields such as movie shooting, advertisement production, and live sports events. Especially in scenes that require complex camera movements, it can provide richer and more dynamic visual effects.

[0169] Figure 7 A schematic structural diagram of a computer system suitable for implementing the method according to an embodiment of the present disclosure is shown.

[0170] As Figure 7 shown, the computer system includes a processing unit, which can execute various methods in the above embodiments according to a program stored in a read-only memory (ROM) or a program loaded from a storage section into a random access memory (RAM). In the RAM, various programs and data required for the operation of the computer system are also stored. The processing unit, the ROM, and the RAM are connected to each other through a bus. An input / output (I / O) interface is also connected to the bus.

[0171] The following components are connected to the I / O interface: an input part including a keyboard, a mouse, etc.; an output part including a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage part including a hard disk, etc.; and a communication part including a network interface card such as a LAN card, a modem, etc. The communication part performs a communication process via a network such as the Internet. A drive is also connected to the I / O interface as needed. A removable medium such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is installed on the drive as needed so that a computer program read from it is installed into the storage part as needed. Among them, the processing unit can be implemented as a processing unit such as a CPU, a GPU, a TPU, an FPGA, an NPU, etc.

[0172] In particular, according to an embodiment of the present disclosure, the method described above can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product that includes a computer program tangibly embodied on a machine-readable medium, the computer program including program code for performing the above method. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part, and / or installed from a removable medium.

[0173] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0174] The units or modules involved in the embodiments described in the present disclosure can be implemented in software or in programmable hardware. The described units or modules can also be provided in a processor, and the names of these units or modules do not constitute a limitation on the units or modules themselves in some cases.

[0175] As another aspect, the present disclosure also provides a computer-readable storage medium, which may be the computer-readable storage medium included in the electronic device or computer system in the above embodiments; or it may exist alone and be a computer-readable storage medium not assembled into the device. The computer-readable storage medium stores one or more programs, and the one or more programs are used by one or more processors to execute the methods described in the present disclosure.

[0176] The above description is only a preferred embodiment of the present disclosure and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of the invention involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. For example, the technical solutions formed by mutually replacing the above features with the technical features (but not limited to) having similar functions disclosed in the present disclosure.

Claims

1. A method for zooming in aerial photography, characterized in that: include: Acquire an initial shooting distance between a shooting device and a shooting object, a specified frame rate, a specified frame zoom ratio, and a progress determination threshold, wherein the shooting device is set on a drone, and the progress determination threshold includes any one of the following: maximum aerial photography zoom time, maximum zoom ratio, and maximum flight speed, wherein the initial shooting distance is the distance between the shooting device and the shooting object when the shooting device and the shooting object are at an initial position; Calculate the fixed acceleration of the drone according to the initial shooting distance, the progress determination threshold, the specified frame rate, and the specified frame zoom factor; While shooting video at the specified frame rate and controlling the shooting device to digitally zoom the shot video according to the specified frame zoom ratio, the drone is controlled to fly away from the shooting object with uniform acceleration along the optical axis direction of the shooting device according to the fixed acceleration starting from the initial shooting distance, thereby realizing aerial push-track zoom. During the aerial push-track zoom process, a progress judgment parameter corresponding to the progress judgment threshold is obtained, and when the progress judgment parameter reaches or exceeds the corresponding progress judgment threshold, the aerial push-track zoom process is ended.

2. The aerial photography zoom method according to claim 1, characterized in that: Also includes: The progress determination parameter includes any one of the aerial photography zoom time, zoom ratio, and flight speed.

3. The method according to claim 2, characterized in that: The maximum aerial photography zoom time is the minimum value between the first maximum aerial photography zoom time and the second maximum aerial photography zoom time, wherein the first maximum aerial photography zoom time is related to the maximum zoom factor, and the second maximum aerial photography zoom time is related to the maximum flight speed.

4. The method according to claim 3, characterized in that: The first maximum aerial zoom time With the maximum zoom ratio The relationship between them satisfies the following formula: ； The second maximum aerial zoom time With the maximum flight speed The relationship between them satisfies the following formula: ； Maximum aerial zoom time With the first maximum aerial zoom time and the second largest aerial zoom time The relationship between them satisfies the following formula: ； Wherein, d is the initial shooting distance, is the specified frame rate, and k is the specified frame scaling factor.

5. The method according to claim 4, characterized in that When the progress determination parameter reaches or exceeds the corresponding progress determination threshold, the aerial photography zoom process is ended, including: When the aerial zoom time t reaches or exceeds the maximum aerial zoom time The aerial photography zoom process ends.

6. The method according to claim 2, characterized in that The method further comprises: When the shooting device is controlled to perform digital zoom on the shot video according to the specified frame zoom ratio, each picture frame of the shot video is enlarged by at least 2 pixels frame by frame; The zoom factor is related to the specified frame magnification.

7. The method according to claim 6, characterized in that: When each picture frame of the captured video is enlarged by at least 2 pixels frame by frame, the specified frame zoom ratio k satisfies the following formula: ； The relationship between the zoom factor n and the specified frame scaling factor k satisfies the following formula: ， ； Wherein, t is the aerial zoom time, is the maximum aerial zoom time, is the maximum zoom factor, is the specified frame rate, w is the width of the first frame of the captured video, h is the height of the first frame of the captured video, and the first frame is the frame captured by the shooting device at the initial shooting distance.

8. The method according to claim 7, characterized in that When the progress determination parameter reaches or exceeds the corresponding progress determination threshold, the aerial photography zoom process is ended, including: When the zoom ratio reaches or exceeds the maximum zoom ratio, the aerial photography zoom process ends.

9. The method according to claim 2, characterized in that: The calculating the fixed acceleration of the drone according to the initial shooting distance, the progress determination threshold, the specified frame rate and the specified frame zoom factor includes calculating the fixed acceleration a according to the following formula: ； Wherein, d is the initial shooting distance, fps is the specified frame rate, k is the specified frame zoom factor, is the maximum aerial zoom time.

10. The method according to claim 9, characterized in that The method of controlling the drone to fly away from the photographed object along the optical axis direction of the photographing device at a uniform acceleration according to the fixed acceleration includes: the drone flying at a fixed speed according to the following To achieve the uniformly accelerated flight: = ； ； in, is the initial flight speed, and t is the aerial zoom time.

11. The method according to claim 10, characterized in that When the progress determination parameter reaches or exceeds the corresponding progress determination threshold, the aerial photography zoom process is ended, including: When the flight speed reaches or exceeds the maximum flight speed, the aerial photography zoom process is ended.

12. An aerial photography zoom system, characterized in that: It includes a main control unit, a flight control module, and a digital zoom module; the flight control module is arranged on the UAV, the digital zoom module is arranged on the shooting device, and the main control unit is connected to the flight control module and the digital zoom module in communication; The main control unit is configured as follows: Acquire an initial shooting distance between a shooting device and a shooting object, a specified frame rate, a specified frame zoom ratio, and a progress determination threshold, wherein the shooting device is set on a drone, and the progress determination threshold includes any one of the following: maximum aerial photography zoom time, maximum zoom ratio, and maximum flight speed, wherein the initial shooting distance is the distance between the shooting device and the shooting object when the shooting device and the shooting object are at an initial position; Calculate the fixed acceleration of the drone according to the initial shooting distance, the progress determination threshold, the specified frame rate, and the specified frame zoom factor; While controlling the camera to shoot a video at the specified frame rate and controlling the digital zoom module to digitally zoom the shot video according to the specified frame zoom ratio, the drone is controlled to fly away from the shooting object along the optical axis direction of the camera at a uniform acceleration starting from the initial shooting distance according to the fixed acceleration, thereby realizing aerial photography push-track zooming; The main control unit includes a progress detection unit configured to obtain a progress determination parameter corresponding to the progress determination threshold during the aerial photography push-track zooming process; The main control unit is further configured to end the aerial photography tracking and zooming process when the progress determination parameter reaches or exceeds the corresponding progress determination threshold.

13. The system according to claim 12, characterized in that The progress determination parameter includes any one of the aerial photography zoom time, zoom ratio, and flight speed.

14. The system according to claim 12, characterized in that The system further comprises a coding output module, wherein the coding output module is arranged on the shooting device; The encoding output module is configured to encode and output the push-track zoom video after completing the aerial photography push-track zoom.

15. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instructions are executed by a processor, the method described in any one of claims 1 to 11 is implemented.

16. A computer program product, comprising computer instructions, which, when executed by a processor, implement the method according to any one of claims 1 to 11.

Images