Flight trajectory planning method, system, storage medium and program product

Abstract

This application provides a flight trajectory planning method, system, storage medium, and program product, relating to the interdisciplinary fields of UAV control, computer vision, and artificial intelligence. The flight trajectory planning method identifies the first scene type corresponding to the target object at the current moment based on the target object's current state data and environmental data. It then further calls upon a target flight trajectory adapted to the first scene type from a preset camera movement strategy library, improving the adaptability of the target flight trajectory to the first scene type. This, in turn, enhances the adaptability of the footage captured by the UAV based on the target flight trajectory to the scene, thereby increasing the expressiveness of the captured footage.

Description

Technical Field

[0001] This application relates to the interdisciplinary fields of unmanned aerial vehicle control, computer vision and artificial intelligence, and in particular to a flight trajectory planning method, system, storage medium and program product. Background Technology

[0002] Currently, with the deep integration of drone and automotive technologies, the application scenarios of vehicle-mounted drones are becoming increasingly diversified, especially in areas such as self-driving tour filming, film and television aerial photography, and tourism.

[0003] In related technologies, the following technology of vehicle-mounted drones mainly relies on GPS positioning or visual tracking algorithms. Specifically, the drone control system uses GPS positioning or visual tracking algorithms to maintain a relatively fixed distance and position between the vehicle-mounted drone and the vehicle, and then controls the vehicle-mounted drone to perform camera movement and shooting based on a preset flight trajectory (such as fixed-point following, circling following, and escort following), thus realizing the basic following function of the vehicle-mounted drone.

[0004] However, the above-mentioned vehicle-mounted drone following technology has the problem of poor adaptability of the footage captured by the drone's camera movement to the scene, resulting in weak image performance. Summary of the Invention

[0005] This application provides a flight trajectory planning method, system, storage medium, and program product to solve the problem that in the implementation of vehicle-mounted drone following technology in related technologies, the footage captured by the drone's camera movement has poor adaptability to the scene and weak image performance.

[0006] In a first aspect, embodiments of this application provide a flight trajectory planning method, including: acquiring the state data and environmental data of a target object to be followed at the current moment; identifying a first scene type corresponding to the target object to be followed at the current moment based on the state data and environmental data; and calling a target flight trajectory adapted to the first scene type based on a preset camera movement strategy library, wherein the target flight trajectory includes flight parameters and camera movement parameters.

[0007] In one possible implementation, the preset camera movement strategy library includes a mapping relationship between preset flight trajectories and preset scene types. Based on the preset camera movement strategy library, a target flight trajectory adapted to the first scene type is planned according to the first scene type, including: determining a target preset scene type consistent with the first scene type in the preset camera movement strategy library; and determining the target flight trajectory based on the mapping relationship and the target preset scene type.

[0008] In one possible implementation, based on state data and environmental data, the first scene type corresponding to the target object at the current moment is identified, including: inputting state data and environmental data into a multimodal data perception module for scene recognition, and obtaining the first scene type output by the multimodal data perception module.

[0009] In one possible implementation, the flight trajectory planning method further includes: analyzing the spatial relationship between the target object and environmental elements, and / or the lighting conditions of the target object, based on state data and environmental data; optimizing and adjusting the target flight trajectory based on the spatial relationship and / or lighting conditions to generate a first flight trajectory.

[0010] In one possible implementation, the flight trajectory planning method further includes: monitoring the second scene type corresponding to the target object in the next moment adjacent to the current moment; in response to the detection that the second scene type is inconsistent with the first scene type, planning a second flight trajectory adapted to the second scene type according to the second scene type; and switching the target flight trajectory to the second flight trajectory based on the point switching principle.

[0011] In one possible implementation, the target flight trajectory is switched to a second flight trajectory based on the point-switching principle, including: extracting the endpoint parameters of the target flight trajectory and the starting parameters of the second flight trajectory based on the point-switching principle; generating a time-continuous, smooth, and abrupt switching trajectory based on the endpoint parameters and the starting parameters; and switching the target flight trajectory to the second flight trajectory based on the switching trajectory.

[0012] In one possible implementation, after calling the target flight trajectory that is compatible with the first scene type, the method further includes: sending the target flight trajectory to the drone's flight control module, so that the flight control module controls the drone to perform flight and camera movement actions according to the target flight trajectory.

[0013] In one possible implementation, during the process of the flight control module controlling the drone to perform flight and camera movement actions according to the target flight trajectory, the method further includes: sending flight control commands to the flight control module, so that the flight control module controls the drone to perform flight and camera movement actions according to the flight control commands and the target flight trajectory.

[0014] In one possible implementation, during the process of the flight control module controlling the drone to perform flight and camera movement actions according to the target flight trajectory, the method further includes: acquiring the drone's currently captured video, which includes camera movement parameters at the time of capture; and editing the currently captured video according to the camera movement parameters.

[0015] In one possible implementation, the flight trajectory planning method further includes: in response to receiving a flight trajectory switching command, switching the target flight trajectory to a third flight trajectory carried in the flight trajectory switching command.

[0016] In one possible implementation, the preset flight trajectories included in the preset camera movement strategy library are constructed in the following way: in response to receiving the user's trajectory drawing operation in the visualization interface, multiple trajectory segments are generated, with the target following object as the center; for each trajectory segment, corresponding preset camera movement parameters are configured for each trajectory segment; the multiple trajectory segments configured with preset camera movement parameters are associated with preset scene types to obtain the preset flight trajectory.

[0017] Secondly, this application provides a flight trajectory planning system, comprising:

[0018] The multimodal data perception module is used to acquire the state data and environmental data of the target object at the current moment, and based on the state data and environmental data, identify the first scene type corresponding to the target object at the current moment;

[0019] The trajectory planning module is used to call the target flight trajectory that is adapted to the first scene type based on the preset camera movement strategy library and the first scene type. The target flight trajectory includes flight parameters and camera movement parameters.

[0020] In one possible implementation, the flight trajectory planning system also includes:

[0021] Dynamic switching decision module: used to monitor the second scene type of the target object in the next moment adjacent to the current moment; and in response to the detection that the second scene type is inconsistent with the first scene type, trigger the trajectory planning module to switch the trajectory.

[0022] In one possible implementation, the multimodal data sensing module is also used to analyze the spatial relationship between the target-following object and environmental elements, and / or the lighting conditions of the target-following object, based on state data and environmental data.

[0023] The trajectory planning module is also used to optimize and adjust the target flight trajectory based on spatial relationships and / or lighting conditions to generate a first flight trajectory.

[0024] Thirdly, this application provides an unmanned aerial vehicle (UAV) system, including: the UAV body, a flight control module, and a flight trajectory planning system as provided in the second aspect above;

[0025] The flight control module is used to control the drone to perform flight and camera movement actions based on the flight trajectory sent by the flight trajectory planning system.

[0026] In one possible implementation, the drone system also includes a drone cabin for providing storage and charging capabilities for the drone.

[0027] In one possible implementation, the drone system also includes a mobile platform on which the drone cabin is mounted.

[0028] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which are executed by a processor to implement the flight trajectory planning method provided in the first aspect.

[0029] Fifthly, embodiments of this application provide a program product comprising computer-executable instructions. When the computer-executable instructions are executed, they implement the flight trajectory planning method provided in the first aspect.

[0030] The flight trajectory planning method, system, storage medium, and program product provided in this application embodiment acquire the state data and environmental data of the target object at the current moment, and identify the first scene type corresponding to the target object at the current moment based on the state data and environmental data. Furthermore, based on a preset camera movement strategy library, it calls a target flight trajectory adapted to the first scene type, where the target flight trajectory includes flight parameters and camera movement parameters. This application improves the adaptability of the target flight trajectory to the first scene type by identifying the first scene type corresponding to the target object at the current moment based on the target object's state data and environmental data, and further calling a target flight trajectory adapted to the first scene type from the preset camera movement strategy library. This enhances the adaptability of the footage captured by the UAV based on the target flight trajectory to the scene, thereby improving the expressiveness of the captured footage. Attached Figure Description

[0031] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0032] Figure 1 This is a schematic diagram of the structure of the flight trajectory planning system provided in the embodiments of this application;

[0033] Figure 2 A flowchart illustrating the flight trajectory planning method provided in this application embodiment. Figure 1 ;

[0034] Figure 3 A flowchart illustrating the flight trajectory planning method provided in this application embodiment. Figure 2 .

[0035] The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0036] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0037] In related technologies, vehicle-mounted drone tracking technology mainly relies on GPS positioning or visual tracking algorithms. Specifically, GPS-based methods calculate the vehicle's position in real time and control the drone to fly within a preset altitude and radius range. This maintains a relatively fixed distance and position between the vehicle-mounted drone and the vehicle, allowing the drone to perform camera movements and capture images based on a preset flight path (such as fixed-point following, circling following, or accompanying following), thus achieving the basic tracking function of the vehicle-mounted drone. Visual tracking methods capture the vehicle's outline using onboard cameras or drone-mounted cameras, and adjust the flight path using image recognition algorithms to maintain a relatively fixed distance and position between the vehicle-mounted drone and the vehicle. This also allows the drone to perform camera movements and capture images based on a preset flight path (such as fixed-point following, circling following, or accompanying following), achieving the basic tracking function of the vehicle-mounted drone.

[0038] However, in the above-mentioned vehicle-mounted drone following technology, the drone becomes disconnected from the scene being filmed during the camera movement and shooting process based on a preset flight trajectory. The drone cannot understand the scene it is filming, such as mountain roads, forests, coastlines, urban building complexes, etc. As a result, it cannot automatically adjust the camera movement according to the real-time scene characteristics to enhance the expressiveness of the shot. There is a problem that the drone's camera movement and shooting footage has poor adaptability to the scene and weak image expressiveness.

[0039] Based on the technical problems existing in related technologies, the embodiments of this application identify the scene type corresponding to the target object at the current moment based on the state data and environmental data of the target object at the current moment, and further call the flight trajectory adapted to the scene type from the preset camera movement strategy library to improve the adaptability of the flight trajectory to the scene type, thereby improving the adaptability of the picture captured by the drone based on the flight trajectory to the scene and enhancing the expressiveness of the captured picture.

[0040] The application scenarios of the embodiments of this application will be described below first.

[0041] The flight trajectory planning method provided in this application is applicable to intelligent aerial photography scenarios using drones in complex and dynamic environments. Specifically, it can be applied to intelligent aerial photography scenarios using vehicle-mounted drones in complex and dynamic environments (such as self-driving tour scenarios and film and television aerial photography scenarios).

[0042] Figure 1 This is a schematic diagram of the flight trajectory planning system provided in an embodiment of this application. Figure 1 As shown, the flight trajectory planning system includes a multimodal data perception module and a trajectory planning module.

[0043] Among them, the multimodal data perception module is used to acquire the state data and environmental data of the target object at the current moment, and based on the state data and environmental data, identify the first scene type corresponding to the target object at the current moment;

[0044] The trajectory planning module is used to call a target flight trajectory that is compatible with the first scene type based on a preset camera movement strategy library and the first scene type. The target flight trajectory includes flight parameters and camera movement parameters.

[0045] For example, the flight trajectory planning system provided in this application embodiment can be loaded onto a mobile platform. The mobile platform includes, but is not limited to, devices such as vehicles and mobile terminals.

[0046] For example, the target object to be followed can be an object that the drone is to follow and photograph, including but not limited to vehicles. This application embodiment does not limit the specific type of the target object to be followed, but can be determined according to the actual application requirements.

[0047] For example, the first scene type can be the macro scene type corresponding to the environment in which the target object is located at the current moment, including but not limited to mountain roads, forest trails, cross-sea bridges, deserts, Gobi, and urban suburbs.

[0048] For example, the multimodal data perception module includes an Artificial Intelligence (AI) model, which is used to identify the first scene type corresponding to the target object at the current moment. The AI ​​model can be a scene classification model based on deep learning. This application embodiment does not limit the type of AI model; it can be determined according to the actual application requirements.

[0049] For example, in one possible implementation, the multimodal data perception module identifies the first scene type corresponding to the target object at the current moment based on the acquired state data and environmental data of the target object being followed. The identified first scene type is then sent to the trajectory planning module, which, based on a preset camera movement strategy library, calls a target flight trajectory that is compatible with the first scene type. The target flight trajectory is then sent to the drone's flight control module, which controls the drone to perform flight and camera movement actions based on the target flight trajectory. This improves the adaptability of the footage captured by the drone based on the target flight trajectory to the scene, enhances the expressiveness of the captured footage, and achieves a fully automatic and highly expressive tracking effect.

[0050] like Figure 1 As shown, optionally, the flight trajectory planning system provided in this application embodiment further includes: a dynamic switching decision module, which is used to monitor the second scene type corresponding to the target following object in the next moment adjacent to the current moment; and in response to detecting that the second scene type is inconsistent with the first scene type, triggering the trajectory planning module to switch the trajectory.

[0051] It is understood that the flight trajectory planning system provided in this application embodiment can continuously monitor the real-time scene type changes of the target object during the flight of the UAV based on the target flight trajectory, and dynamically trigger trajectory switching when the scene type changes are detected, thereby improving the adaptability of the UAV's flight trajectory to the scene type of the target object.

[0052] Optionally, the multimodal data perception module is also used to analyze the spatial relationship between the target-following object and environmental elements, and / or the lighting conditions of the target-following object, based on state data and environmental data.

[0053] Correspondingly, the trajectory planning module is also used to optimize and adjust the target flight trajectory based on spatial relationships and lighting conditions to generate the first flight trajectory.

[0054] It is understood that the flight trajectory planning system provided in this application embodiment can be an intelligent closed-loop system of "perception-decision-execution". On the one hand, the flight trajectory planning system analyzes the target object and its surrounding environment in real time through a multimodal data perception module, identifies the first scene type corresponding to the target object at the current moment, and intelligently calls the target flight trajectory that is adapted to the first scene type based on a preset camera movement strategy library through the trajectory planning module. On the other hand, during the flight of the UAV based on the target flight trajectory, the dynamic switching decision module continuously monitors the real-time scene type changes of the target object, and dynamically triggers trajectory switching when a scene type change is detected, so as to achieve a fully automatic and high-performance tracking effect.

[0055] Optionally, the flight trajectory planning system provided in this application embodiment may further include an image data module. Figure 1 (Not shown in the image), this image data module is used to store and create images captured by the drone during flight and camera movement, as well as to create videos based on the captured images.

[0056] For example, in the flight trajectory planning system provided in this application embodiment, the identification of the first scene type corresponding to the target object at the current moment, the analysis of the spatial relationship between the target object and environmental elements and / or the lighting conditions of the target object, and the trajectory planning of the target flight trajectory adapted to the first scene type can be uploaded to the vehicle computing unit or cloud server for processing.

[0057] The following is based on Figure 1 The flight trajectory planning system shown is the execution entity. The specific implementation of the flight trajectory planning method provided in this application embodiment will be described in detail with reference to specific embodiments.

[0058] Figure 2 A flowchart illustrating the flight trajectory planning method provided in this application embodiment. Figure 1 .like Figure 2 As shown, a specific implementation of this flight trajectory planning method may include the following steps:

[0059] S201, Obtain the current state and environment data of the target object being followed.

[0060] For example, the target object to be followed can be an object that the drone is to follow and photograph, including but not limited to vehicles. This application embodiment does not limit the specific type of the target object to be followed, but can be determined according to the actual application requirements.

[0061] For example, when the target object to be followed is a vehicle, the current state data of the target object to be followed includes, but is not limited to, the vehicle's current speed, orientation, and location information; the current environmental data of the target object to be followed includes, but is not limited to, the curvature and slope of the road, the distribution of landmarks (such as the location information of iconic buildings), the distribution of natural landscapes, and the direction of light.

[0062] For example, the current state data and environmental data of the target object being followed can be multimodal data collected based on vehicle sensors (such as vehicle cameras, dashcams, LiDAR, inertial measurement units (IMUs), global positioning systems (GPS)), vehicle-mounted drone cabin cameras, and drone aerial cameras.

[0063] S202, Based on the state data and environmental data, identify the first scene type corresponding to the target object at the current moment.

[0064] For example, the first scene type can be the macro scene type corresponding to the environment in which the target object is located at the current moment, including but not limited to mountain roads, forest trails, cross-sea bridges, deserts, Gobi, and urban suburbs.

[0065] In this step, one possible implementation is to input the state data and environmental data into the AI ​​model set in the multimodal data perception module to identify the scene type and obtain the first scene type output by the AI ​​model.

[0066] Another possible implementation of this step is to input the state data and environmental data into the AI ​​model set in the multimodal data perception module to identify the scene type, obtain multiple different scene types output by the AI ​​model, and the confidence level corresponding to each scene type. The multimodal data perception module determines the scene type with the highest confidence level as the first scene type and outputs the first scene type.

[0067] S203, based on the preset camera movement strategy library, according to the first scene type, call the target flight trajectory that is compatible with the first scene type. The target flight trajectory includes flight parameters and camera movement parameters.

[0068] For example, the preset camera movement strategy library can be stored as described above. Figure 1 The trajectory planning module in the flight trajectory planning system shown.

[0069] For example, the preset camera movement strategy library stores a variety of preset flight trajectories bound to preset scene types, that is, each preset scene type corresponds to a preset flight trajectory. Among them, the preset flight trajectories include, but are not limited to, intelligent film camera movement trajectory, human-vehicle group photo trajectory, camping mode trajectory, Gobi Desert crossing trajectory, coastal highway trajectory, etc.; the preset scene types include, but are not limited to, mountain roads, forest trails, cross-sea bridges, deserts, Gobi, urban suburbs, etc.

[0070] For example, each preset flight trajectory can be customized based on the corresponding preset scene type, and each preset flight trajectory can be formed by a combination of at least one camera movement strategy.

[0071] Understandably, compared to related technologies where drone flight trajectories are typically preset (such as fixed-point following or circling), lacking creativity and dynamic variation, the shooting modes of drones based on preset flight trajectories are relatively limited. This application's embodiments, by customizing preset flight trajectories for preset scene types, and with each preset flight trajectory potentially formed by a combination of at least one camera movement strategy, effectively enhance the diversity of shooting modes.

[0072] It is understandable that the target flight trajectory can be the relative running trajectory between the UAV and the target object being followed, that is, the UAV flies with the target object being followed as a reference during flight.

[0073] For example, the target flight trajectory can be any one of the following: intelligent film movement trajectory, human-vehicle group photo trajectory, camping mode trajectory, Gobi Desert crossing trajectory, or coastal highway trajectory.

[0074] For example, flight parameters can be parameters used to control the drone's own flight motion, including but not limited to flight position, flight altitude, flight speed, flight attitude (pitch angle, roll angle, yaw angle), acceleration, and angular velocity; camera movement parameters can be parameters used to control the drone's shooting angle and lens movement, including gimbal movement parameters and drone camera shooting parameters. Among them, gimbal movement parameters include but are not limited to gimbal pitch angle, gimbal yaw angle, shooting angle, following angle, and camera movement type (such as thrust, orbit, follow, low-angle shot, high-angle shot), while drone camera shooting parameters include but are not limited to aperture, exposure, lens focal length, and zoom ratio.

[0075] It is understood that in this embodiment of the application, the target flight trajectory adapted to the first scene type takes into account the flight safety of the UAV, the stability of the captured images, and the artistic expression of the UAV in the first scene type.

[0076] This embodiment of the application acquires the state data and environmental data of the target object at the current moment, and identifies the first scene type corresponding to the target object at the current moment based on the state data and environmental data. Furthermore, based on a preset camera movement strategy library, it calls a target flight trajectory adapted to the first scene type, the target flight trajectory including flight parameters and camera movement parameters. This embodiment of the application improves the adaptability of the target flight trajectory to the first scene type by identifying the first scene type corresponding to the target object at the current moment based on the state data and environmental data of the target object, and further calling a target flight trajectory adapted to the first scene type from the preset camera movement strategy library. This enhances the adaptability of the footage captured by the UAV based on the target flight trajectory to the scene, thereby enhancing the expressiveness of the captured footage.

[0077] It is understandable that, compared to related technologies where the system primarily perceives the position and speed of the vehicle, it lacks a deep semantic understanding of the surrounding environment (such as road conditions, terrain, landmarks, and lighting conditions), resulting in weak scene perception capabilities. This application's embodiment enhances scene perception capabilities by using a multimodal data perception module to identify the scene type based on the target object's current state data and environmental data.

[0078] Optionally, the preset camera movement strategy library provided in this application embodiment includes a mapping relationship between preset flight trajectories and preset scene types.

[0079] For example, each preset scene type corresponds to a preset flight trajectory. The preset flight trajectories and preset scene types are similar to those described above, and will not be repeated here.

[0080] Optionally, in step S203, based on a preset camera movement strategy library, a specific implementation method for calling a target flight trajectory that is compatible with the first scene type according to the first scene type can be: in the preset camera movement strategy library, determine a target preset scene type that is consistent with the first scene type; based on the mapping relationship, determine the target flight trajectory according to the target preset scene type.

[0081] For example, the target flight trajectory can be a preset flight trajectory in the preset camera movement strategy library that corresponds to the preset scene type of the target.

[0082] In this embodiment of the application, a target preset scene type that is consistent with the first scene type is determined in the preset camera movement strategy library. Furthermore, based on the mapping relationship, the target flight trajectory is determined according to the target preset scene type, thereby improving the adaptability of the target flight trajectory and the first scene type. This improves the adaptability of the footage captured by the UAV based on the target flight trajectory and the scene, and enhances the expressiveness of the captured footage.

[0083] Optionally, step S202, which identifies the first scene type corresponding to the target following object at the current moment based on the state data and environment data, can be implemented by inputting the state data and environment data into the multimodal perception module for scene recognition, and obtaining the first scene type output by the multimodal perception module.

[0084] The specific implementation of this embodiment is similar to that described above, and will not be repeated here.

[0085] In one possible implementation, the flight trajectory planning method provided in this application embodiment may further include: analyzing the spatial relationship between the target object and environmental elements, and / or the lighting conditions of the target object, based on state data and environmental data; optimizing and adjusting the target flight trajectory based on the spatial relationship and / or lighting conditions to generate a first flight trajectory.

[0086] For example, environmental elements are similar to the environmental data mentioned above, and will not be described again here.

[0087] For example, spatial relationships include, but are not limited to, the relative positional relationship, relative distance relationship, relative orientation relationship or relative orientation relationship (such as the orientation or orientation of the target object relative to the environmental element), and relative height relationship between the target object and environmental elements.

[0088] For example, the lighting conditions of the target object include the state in which the target object is illuminated by light, such as the target object being partially illuminated, the target object being completely illuminated (i.e., the target object being too brightly lit), and the target object not being illuminated (i.e., the target object being too dimly lit).

[0089] The specific type of the first flight trajectory is similar to that of the target flight trajectory mentioned above, and will not be repeated here.

[0090] For example, in some embodiments: state data and environmental data are input into the above... Figure 1 In the AI ​​model of the multimodal data perception module shown, the AI ​​model analyzes the spatial relationship between the target object and environmental elements based on the shooting safety and stability of the drone, and obtains the spatial relationship between the target object and environmental elements output by the AI ​​model. The multimodal data perception module further optimizes and adjusts the flight parameters and / or camera movement parameters corresponding to the target flight trajectory based on the spatial relationship, and generates a first flight trajectory, so that the drone can perform flight and camera movement shooting actions based on the first flight trajectory.

[0091] For example, in some embodiments, state data and environmental data are input into the above... Figure 1In the AI ​​model of the multimodal data perception module shown, the AI ​​model analyzes the lighting conditions of the target object based on the shooting safety and stability of the drone, and obtains the lighting conditions of the target object output by the AI ​​model. The multimodal data perception module further optimizes and adjusts the flight parameters and / or camera movement parameters corresponding to the target flight trajectory based on the lighting conditions, and generates a first flight trajectory, so that the drone can perform flight and camera movement shooting actions based on the first flight trajectory.

[0092] For example, in some embodiments: state data and environmental data are input into the above... Figure 1 In the AI ​​model of the multimodal data perception module shown, the AI ​​model analyzes the spatial relationship between the target object and environmental elements and the lighting conditions of the target object based on the shooting safety and stability of the drone. The AI ​​model outputs the spatial relationship between the target object and environmental elements and the lighting conditions of the target object. The multimodal data perception module further optimizes and adjusts the flight parameters and / or camera movement parameters corresponding to the target flight trajectory based on the spatial relationship and lighting conditions to generate a first flight trajectory, so that the drone can perform flight and camera movement shooting actions based on the first flight trajectory.

[0093] It is understood that in the flight trajectory planning method provided in this application embodiment, when the UAV is in the following state, it can analyze the spatial relationship between the target object and environmental factors, and / or the lighting conditions of the target object, and optimize and adjust the target flight trajectory based on the spatial relationship and / or lighting conditions to intelligently generate a continuous dynamic flight trajectory, namely the first flight trajectory, so as to complete the high, low and near distance shooting of the target object.

[0094] In this embodiment, by analyzing the spatial relationship between the target object and environmental elements and / or the lighting conditions of the target object based on state data and environmental data, the target flight trajectory is further optimized and adjusted based on the spatial relationship and / or lighting conditions to generate a first flight trajectory, thereby improving the shooting safety and stability of the UAV during flight and camera movement based on the first flight trajectory.

[0095] Compared to related technologies, where users typically need to manually select the follow mode before the drone begins flight, and once the drone starts executing its flight path, it cannot intelligently and smoothly switch its trajectory based on changes in the environment of the target object during flight, even if the scene type corresponding to the target object changes drastically at the current moment, the system cannot switch the current flight path to a new flight path that is more suitable for the scene type corresponding to the target object at the current moment in a real-time and smooth manner, resulting in poor scene adaptability and interactivity of the drone. In the flight trajectory planning method provided in this application embodiment, the drone can trigger a flight trajectory switch based on changes in the scene type corresponding to the real-time environment of the target object during flight, thereby improving the scene adaptability and interactivity of the drone.

[0096] The following is combined Figure 3 This application provides a detailed description of a specific implementation method for switching the flight trajectory of a UAV based on changes in the real-time scene type of the target object being followed, within the flight trajectory planning method provided in this embodiment.

[0097] Figure 3 A flowchart illustrating the flight trajectory planning method provided in this application embodiment. Figure 2 .like Figure 3 As shown, a specific implementation of this flight trajectory planning method may include the following steps:

[0098] S301, the second scene type corresponding to the next moment adjacent to the current moment for the monitored target and the object being followed.

[0099] The specific types of the second scene type are similar to those of the first scene type mentioned above, and will not be repeated here.

[0100] In this step, one possible implementation is as follows: The multimodal data perception module in the flight trajectory planning system acquires the state data and environmental data of the target object at the next moment adjacent to the current moment, and inputs the state data and environmental data of the next moment into the AI ​​model in the multimodal data perception module for scene type identification, and obtains the second scene type corresponding to the next moment output by the AI ​​model. The multimodal data perception module monitors in real time whether the second scene type corresponding to the next moment is consistent with the first scene type corresponding to the current moment. When the second scene type is consistent with the first scene type, the UAV continues to execute the target flight trajectory adapted to the first scene type. When the second scene type is inconsistent with the first scene type, the UAV switches to a new flight trajectory adapted to the second scene type, i.e., the second flight trajectory.

[0101] S302, in response to the detection that the second scene type is inconsistent with the first scene type, plans a second flight trajectory that is compatible with the second scene type.

[0102] The specific type of the second flight trajectory is similar to that of the target flight trajectory mentioned above, and will not be repeated here.

[0103] In this step, one possible implementation is as follows: in response to detecting that the second scene type is inconsistent with the first scene type, a preset scene type consistent with the second scene type is determined in the preset camera movement strategy library, and based on the mapping relationship between the preset flight trajectory and the preset scene type, the second flight trajectory is determined according to the preset scene type consistent with the second scene type.

[0104] S303, based on the point-switching principle, switches the target flight trajectory to the second flight trajectory.

[0105] For example, the point switching principle can be based on a dual-circle compass. The dual-circle compass includes an inner circular track and an outer circular track. Both the inner and outer circular tracks include a preset number of atomic points. Each atomic point corresponds to a flight path, i.e., a camera movement type (such as orbiting, soaring, following, etc.). This flight path includes a starting point and an ending point. This application embodiment does not limit the number or position of atomic points in the inner and outer circular tracks; they can be determined according to actual application requirements.

[0106] For example, both the target flight trajectory and the second flight trajectory can be composed of multiple atomic points set on the inner circular trajectory and / or the outer circular trajectory. That is, both the target flight trajectory and the second flight trajectory can contain a variety of different camera movement types, and the atomic points corresponding to each camera movement type are different.

[0107] It is understood that, in this embodiment of the application, when switching the target flight trajectory to the second flight trajectory based on the point switching principle of the dual-circle compass, the switching can be performed between the inner and outer circular trajectories of the dual-circle compass, or between different atomic points within the same circular trajectory, based on the positions of the atomic points included in the target flight trajectory and the positions of the atomic points included in the second flight trajectory. Each switching operation is based on a smooth transition between the starting and ending points of the atomic points to ensure the flight safety, stability, and continuity of camera movement during the switching process of the UAV, thereby improving the UAV's adaptability to different scene types.

[0108] Optionally, one possible implementation of this step is as follows: based on the point switching principle, extract the endpoint parameters of the target flight trajectory and the starting parameters of the second flight trajectory; generate a time-continuous, smooth, and abrupt switching trajectory based on the endpoint parameters and the starting parameters; and switch the target flight trajectory to the second flight trajectory based on the switching trajectory.

[0109] For example, the endpoint parameter of the target flight trajectory can be the endpoint of the atomic point currently flying in the target flight trajectory, and the starting point parameter of the second flight trajectory can be the starting point of the first atomic point in the second flight trajectory.

[0110] In this embodiment, one possible implementation is as follows: the trajectory generator in the trajectory planning module extracts the endpoint parameters of the target flight trajectory and the starting parameters of the second flight trajectory based on the point switching principle. Based on the endpoint and starting parameters, and considering the drone's flight safety, image stability, and the artistic expressiveness of the captured footage, it generates a smooth, continuous trajectory that includes flight parameters and camera movement parameters. Furthermore, by switching the trajectory, the target flight trajectory is switched to the second flight trajectory. The flight parameters and camera movement parameters are similar to those described above and will not be repeated here.

[0111] It is understood that the implementation method of switching the target flight trajectory to the second flight trajectory provided in this application embodiment can avoid shaking of the captured image during the flight of the UAV and ensure the stability of the captured image.

[0112] It is understood that in the flight trajectory planning method provided in this application embodiment, the flight trajectory planning system can intelligently identify and monitor the scene type corresponding to the real-time environment of the target following object through AI capabilities, and realize the dynamic switching of the drone flight trajectory when the scene type is detected to change.

[0113] This embodiment monitors the second scene type corresponding to the target object in the next adjacent moment. When the second scene type is found to be inconsistent with the first scene type, a second flight trajectory adapted to the second scene type is planned. Furthermore, based on the point-switching principle, the target flight trajectory is switched to the second flight trajectory. This achieves scene trajectory switching triggered by changes in the scene type corresponding to the real-time environment of the target object, improving the scene adaptability and interactivity of the drone. Simultaneously, it can cope with complex and dynamically changing environments where the target object is located, ensuring the safe, continuous, and high-quality completion of drone shooting missions and improving the drone's scene adaptability.

[0114] Compared to related technologies, the system cannot intelligently push flight trajectories to users based on scene characteristics, resulting in a monotonous user experience with the drone's functions, lacking novelty and surprise, and leading to a relatively simple user experience. This application's embodiments address this by, on the one hand, pre-setting flight trajectories adapted to different preset scene types, and by identifying the scene type corresponding to the real-time environment of the target object to plan a flight trajectory adapted to that scene type; on the other hand, by monitoring the scene type corresponding to the real-time environment of the target object in real time, and triggering intelligent switching of the flight trajectory when a change in scene type is detected, thereby enriching the drone's functional experience, enhancing novelty and surprise, and ultimately improving the diversity of the user experience.

[0115] Optionally, in the flight trajectory planning method provided in this application embodiment, after calling the target flight trajectory adapted to the first scene type, it further includes: sending the target flight trajectory to the flight control module of the UAV, so that the flight control module controls the UAV to perform flight and camera movement actions according to the target flight trajectory.

[0116] In this embodiment, one possible implementation is as follows: the flight trajectory planning system sends the target flight trajectory to the flight control module of the UAV, so that the flight control module controls the UAV to perform flight and camera movement actions based on the flight parameters and camera movement parameters included in the target flight trajectory.

[0117] In this embodiment, after calling the target flight trajectory that is compatible with the first scene type, the target flight trajectory is sent to the flight control module of the drone. The flight control module controls the drone to perform flight and camera movement actions according to the target flight trajectory. In this way, the AI ​​model simulates the camera movement thinking of a professional photographer, making the video generated by the drone dynamic, beautifully composed, with a strong cinematic feel and storytelling, effectively enhancing the expressiveness of the captured images.

[0118] It should be noted that in the flight trajectory planning method provided in this application embodiment, if the flight control module of the UAV controls the UAV to perform flight and camera movement actions according to the target flight trajectory, and no flight trajectory switching occurs, the flight control module controls the UAV to perform flight and camera movement actions according to the target flight trajectory until the UAV flies to the end point of the last atomic point in the target flight trajectory. At this point, the flight control module controls the UAV to only perform flight following of the target object and no longer perform camera movement actions.

[0119] Optionally, the flight trajectory planning method provided in this application embodiment further includes, during the process of the flight control module controlling the UAV to perform flight and camera movement shooting actions according to the target flight trajectory, sending flight control commands to the flight control module so that the flight control module controls the UAV to perform flight and camera movement shooting actions according to the flight control commands and the target flight trajectory.

[0120] For example, flight control commands can be applied by the user to load data as described above. Figure 1 The interactive interface of the flight trajectory planning system shown is generated by the vehicle or mobile terminal.

[0121] For example, flight control commands are used to instruct the drone to perform flight and camera movement actions according to the user's manual control.

[0122] In this embodiment, one possible implementation is as follows: during the process of the flight control module controlling the drone to perform flight and camera movement actions according to the target flight trajectory, in response to the above... Figure 1 The flight trajectory planning system shown receives flight control commands generated by the user through the interactive interface. The flight trajectory planning system sends the flight control commands to the UAV's flight control module, causing the flight control module to perform flight and camera movement actions according to the user's manual control. After the manual control ends, the trajectory planning module in the flight trajectory planning system automatically plans a switching trajectory to connect the manual control and the target flight trajectory. The flight control module then controls the UAV to perform flight and camera movement actions based on this switching trajectory, so as to automatically restore the flight and camera movement actions based on the target flight trajectory.

[0123] In this embodiment, during the process of the flight control module controlling the drone to perform flight and camera movement actions according to the target flight trajectory, flight control commands are sent to the flight control module, so that the flight control module controls the drone to perform flight and camera movement actions according to the flight control commands and the target flight trajectory, thereby improving user experience and interactivity.

[0124] Optionally, the flight trajectory planning method provided in this application embodiment further includes, during the process of the flight control module controlling the UAV to perform flight and camera movement shooting actions according to the target flight trajectory: acquiring the current shooting video of the UAV, the current shooting video including the camera movement parameters at the time of shooting, and editing the current shooting video according to the camera movement parameters.

[0125] The camera movement parameters are similar to those described above, and will not be repeated here.

[0126] For example, the currently captured video can be stored in the image data module of the flight trajectory planning system.

[0127] For example, the currently captured video can also be labeled with the corresponding scene type and flight path.

[0128] In this embodiment, one possible implementation is as follows: the image data module in the flight trajectory planning system acquires the current video captured by the UAV and preprocesses the current video (e.g., divides the current video into different video clips according to the camera movement parameters) to obtain the preprocessed video. The preprocessed video is then input into a video editing application to perform subsequent editing and optimization operations to complete the editing of the current video.

[0129] For example, in some embodiments, the currently captured video may also include drone flight data (such as flight distance, flight time, flight speed, etc.) and drone status data.

[0130] Optionally, the flight trajectory planning method provided in this application embodiment further includes: in response to receiving a flight trajectory switching instruction, switching the target flight trajectory to a third flight trajectory carried in the flight trajectory switching instruction.

[0131] For example, the flight path switching command can be applied by the user to a loaded object as described above. Figure 1 The interactive interface of the flight trajectory planning system shown is generated by the vehicle or mobile terminal.

[0132] For example, the third flight path can be one or more preset flight paths that the user manually selects from a preset camera movement strategy library.

[0133] In this embodiment, one possible implementation is: in response to the above... Figure 1 The trajectory planning module in the flight trajectory planning system shown receives the flight trajectory switching command generated by the user through the interactive interface. The trajectory planning module automatically plans a switching trajectory to connect the third flight trajectory and the target flight trajectory, and switches the target flight trajectory to the third flight trajectory through this switching trajectory.

[0134] It is understood that in the flight trajectory planning method provided in this application embodiment, during the process of the UAV's flight control module controlling the UAV to perform flight and camera movement shooting actions, its flight trajectory can be automatically switched by monitoring the scene type corresponding to the real-time environment of the target object through the flight trajectory planning system, or it can be manually switched by the flight trajectory switching command sent by the user.

[0135] Optionally, in the flight trajectory planning method provided in this application embodiment, the preset flight trajectory included in the preset camera movement strategy library can be constructed in the following way: in response to receiving the user's trajectory drawing operation on the visualization interface, multiple trajectory segments are generated, the visualization interface being centered on the target following object; for each trajectory segment, corresponding preset camera movement parameters are configured for each trajectory segment; the multiple trajectory segments configured with preset camera movement parameters are associated with preset scene types to obtain the preset flight trajectory.

[0136] For example, the visualization interface can be a three-dimensional coordinate system interface.

[0137] For example, a user's trajectory drawing operation can be a line or vertex drawn by the user in a visualization interface.

[0138] It is understandable that each of the multiple trajectories corresponds to a certain type of camera movement.

[0139] In this embodiment, one possible implementation is as follows: in response to receiving a trajectory drawing operation formed by the user in the visualization interface by drawing lines or fixing points, multiple trajectories are generated, and for each of the multiple trajectories, corresponding preset camera movement parameters are configured for each trajectories. Furthermore, the multiple trajectories configured with preset camera movement parameters are associated with preset scene types to obtain preset flight trajectories.

[0140] Compared to related technologies, flight trajectories are mostly simple paths, unable to simulate the complex and continuous dynamic camera movements of professional photographers based on scene type and target objects, resulting in rigid flight trajectory generation. In this application, in response to a user's trajectory drawing operation received on a visual interface, multiple trajectory segments are generated. For each of these segments, corresponding preset camera movement parameters are configured. Furthermore, the multiple trajectories configured with preset camera movement parameters are associated with preset scene types to obtain preset flight trajectories. This achieves the simulation of complex and continuous dynamic camera movements of professional photographers based on scene type and target objects, effectively improving the flexibility of the preset flight trajectory.

[0141] Optionally, in some embodiments, the preset flight trajectories included in the preset camera movement strategy library can also be constructed in the following way: the flight trajectory planning system records the user's feedback information on the automatically generated shooting video (such as the original video download frequency, the editing usage frequency, etc.), and continuously optimizes and personalizes the preset flight trajectories in the preset camera movement strategy library through deep learning algorithms (such as reinforcement learning algorithms) to construct preset flight trajectories that better meet the user's aesthetic preferences.

[0142] In summary, the flight trajectory planning method provided in this application has the following beneficial effects:

[0143] 1) By identifying the scene type corresponding to the target object at the current moment based on the state data and environmental data of the target object, and further calling the flight trajectory that is compatible with the scene type in the preset camera movement strategy library, the adaptability of the flight trajectory and the scene type is improved, thereby improving the adaptability of the footage captured by the drone based on the flight trajectory and the scene, and enhancing the expressiveness of the captured footage.

[0144] 2) After calling the target flight trajectory that is compatible with the first scene type, the target flight trajectory is sent to the drone's flight control module, so that the flight control module controls the drone to perform flight and camera shooting actions according to the target flight trajectory. In this way, the AI ​​model simulates the camera movement thinking of a professional photographer, making the video generated by the drone dynamic, beautifully composed, with a strong cinematic feel and storytelling, effectively enhancing the expressiveness of the shot.

[0145] 3) By monitoring the target object's second scene type in the next moment adjacent to the current moment, and when the second scene type is found to be inconsistent with the first scene type, a second flight trajectory adapted to the second scene type is planned. Furthermore, based on the point-switching principle, the target flight trajectory is switched to the second flight trajectory. This achieves scene type switching triggered by changes in the real-time environment of the target object, improving the drone's scene adaptability and interactivity. Simultaneously, it can cope with complex and dynamically changing environments where the target object is located, ensuring the safe, continuous, and high-quality completion of drone shooting missions, and improving the drone's scene adaptability.

[0146] 4) Achieve one-click intelligent scene creation of blockbuster effects, significantly lower the threshold for professional aerial photography, improve user experience, and realize the intelligence and automation of the drone aerial photography process;

[0147] 5) It deeply integrates computer vision, deep learning, path planning and control theory, providing a new paradigm for the application of intelligent UAV flight trajectory planning in complex dynamic environments.

[0148] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0149] This application also provides an unmanned aerial vehicle (UAV) system, including: an UAV body, a flight control module, and a flight trajectory planning system as described in the above embodiments;

[0150] The flight control module is used to control the drone to perform flight and camera movement actions based on the flight trajectory sent by the flight trajectory planning system.

[0151] Optionally, in one possible implementation, the drone system provided in this application embodiment further includes a drone cabin for providing storage and charging functions for the drone.

[0152] Optionally, in one possible implementation, the unmanned aerial vehicle (UAV) system provided in this application embodiment further includes a mobile platform, on which the UAV cabin is mounted.

[0153] For example, a mobile platform can be a vehicle, a robotic platform (such as a server robot, an exploration robot, a scientific research robot, etc.), a drone, or other equipment, but is not limited to these.

[0154] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0155] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0156] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0157] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0158] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0159] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0160] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0161] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0162] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0163] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A flight trajectory planning method, characterized by, include: Obtain the current state and environment data of the target object being followed; Based on the state data and the environment data, identify the first scene type corresponding to the target following object at the current moment; Based on a preset camera movement strategy library, a target flight trajectory adapted to the first scene type is invoked according to the first scene type. The target flight trajectory includes flight parameters and camera movement parameters.

2. The flight trajectory planning method according to claim 1, characterized in that, The preset camera movement strategy library includes a mapping relationship between preset flight trajectories and preset scene types. The step of planning a target flight trajectory adapted to the first scene type based on the preset camera movement strategy library includes: In the preset camera movement strategy library, a target preset scene type that matches the first scene type is determined; Based on the mapping relationship, the target flight trajectory is determined according to the target preset scene type.

3. The flight trajectory planning method according to claim 1, characterized in that, The step of identifying the first scene type corresponding to the target following object at the current moment based on the state data and the environment data includes: The state data and the environment data are input into the multimodal data perception module for scene recognition, and the first scene type output by the multimodal data perception module is obtained.

4. The flight trajectory planning method according to any one of claims 1 to 3, characterized in that, Also includes: Based on the state data and the environment data, analyze the spatial relationship between the target object and environmental elements, and / or the lighting conditions of the target object; Based on the spatial relationships and / or the lighting conditions, the target flight trajectory is optimized and adjusted to generate a first flight trajectory.

5. The flight trajectory planning method according to any one of claims 1 to 3, characterized in that, Also includes: Monitor the second scene type of the target object in the next time step adjacent to the current time step; In response to the detection that the second scene type is inconsistent with the first scene type, a second flight trajectory adapted to the second scene type is planned according to the second scene type; Based on the point switching principle, the target flight trajectory is switched to the second flight trajectory.

6. The flight trajectory planning method according to claim 5, characterized in that, The step of switching the target flight trajectory to the second flight trajectory based on the point-switching principle includes: Based on the aforementioned point switching principle, the endpoint parameters of the target flight trajectory and the starting point parameters of the second flight trajectory are extracted. Based on the endpoint parameter and the starting point parameter, a switching trajectory that is continuous in time, smooth, and without abrupt changes is generated; According to the switching trajectory, the target flight trajectory is switched to the second flight trajectory.

7. The flight trajectory planning method according to any one of claims 1 to 3, characterized in that, After invoking the target flight trajectory adapted to the first scenario type, the process also includes: The target flight trajectory is sent to the flight control module of the drone, so that the flight control module controls the drone to perform flight and camera movement actions according to the target flight trajectory.

8. The flight trajectory planning method according to claim 7, characterized in that, During the process of the flight control module controlling the UAV to perform flight and camera movement actions according to the target flight trajectory, the following is also included: The system sends flight control commands to the flight control module, enabling the flight control module to control the UAV to perform flight and camera movement actions based on the flight control commands and the target flight trajectory.

9. The flight trajectory planning method according to claim 7, characterized in that, During the process of the flight control module controlling the UAV to perform flight and camera movement actions according to the target flight trajectory, the following is also included: Acquire the currently captured video of the drone, the currently captured video including the camera movement parameters at the time of capture; Edit the currently captured video according to the camera movement parameters.

10. The flight trajectory planning method according to any one of claims 1 to 3, characterized in that, Also includes: In response to receiving a flight trajectory switching command, the target flight trajectory is switched to the third flight trajectory carried in the flight trajectory switching command.

11. The flight trajectory planning method according to any one of claims 1 to 3, characterized in that, The preset flight trajectories included in the preset camera movement strategy library are constructed in the following way: In response to receiving a trajectory drawing operation from a user in a visualization interface, multiple trajectory segments are generated, wherein the visualization interface is centered on the target object being followed. For each of the multiple trajectory segments, configure corresponding preset camera movement parameters for each trajectory segment; The preset flight trajectory is obtained by associating multiple trajectories with preset camera movement parameters with preset scene types.

12. A flight trajectory planning system, characterized in that, include: The multimodal data perception module is used to acquire the state data and environmental data of the target object at the current moment, and based on the state data and environmental data, identify the first scene type corresponding to the target object at the current moment; The trajectory planning module is used to call a target flight trajectory that is compatible with the first scene type based on a preset camera movement strategy library and the first scene type. The target flight trajectory includes flight parameters and camera movement parameters.

13. The flight trajectory planning system according to claim 12, characterized in that, Also includes: Dynamic switching decision module: used to monitor the second scene type corresponding to the target following object in the next moment adjacent to the current moment; In response to the detection that the second scene type is inconsistent with the first scene type, the trajectory planning module is triggered to switch trajectories.

14. The flight trajectory planning system according to claim 12 or 13, characterized in that, The multimodal data sensing module is also used to analyze the spatial relationship between the target following object and environmental elements, and / or the lighting conditions of the target following object, based on the state data and the environmental data. The trajectory planning module is also used to optimize and adjust the target flight trajectory based on the spatial relationship and the lighting conditions to generate a first flight trajectory.

15. An unmanned aerial vehicle (UAV) system, characterized in that, include: The unmanned aerial vehicle (UAV) body, the flight control module, and the flight trajectory planning system as described in any one of claims 12 to 14; The flight control module is used to control the UAV to perform flight and camera movement actions according to the flight trajectory sent by the flight trajectory planning system.

16. The unmanned aerial vehicle system according to claim 15, characterized in that, Also includes: The drone cabin is used to provide storage and charging functions for the drone.

17. The unmanned aerial vehicle system according to claim 16, characterized in that, Also includes: A mobile platform, on which the drone cabin is mounted.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the flight trajectory planning method as described in any one of claims 1 to 11.

19. A computer program product, characterized in that, include: A computer program, which, when executed by a processor, implements the flight trajectory planning method as described in any one of claims 1 to 11.

Images