Route editing method, electronic device, system and storage medium

Abstract

Disclosed in the present application are a route editing method, an electronic device, a system and a storage medium. The method comprises: on the basis of a display of a head-mounted device, displaying a three-dimensional spatial map of a selected position; and in response to an interactive operation on a remote controller, performing route editing in the three-dimensional spatial map on the basis of an interactive interface, so as to obtain a flight route of a movable platform. In this way, route editing can be performed in a three-dimensional spatial map on the basis of an interactive operation on a remote controller, thereby implementing immersive route editing, enhancing the intuitiveness of route editing, and effectively improving the editing efficiency compared with test flight methods in the real world; in addition, due to a high degree of visualization, the relationship between a route and buildings or terrains in the three-dimensional spatial map can be effectively viewed, thereby improving the reliability of route editing.

Description

Route editing methods, electronic devices, systems and storage media Technical Field

[0001] This application relates to the field of unmanned aerial vehicle (UAV) control, and more particularly to a flight path editing method, electronic device, system, and storage medium. Background Technology

[0002] With the development of technology, drones have been widely used in various fields such as aerial photography, surveying and mapping, and search and rescue. Efficient flight path planning is undoubtedly a prerequisite for ensuring the smooth progress of missions.

[0003] In related technologies, drones often need to be test-flown in the real world to edit flight paths, which affects the efficiency of flight path editing. If flight path planning software is used for flight path editing, it is difficult to closely match the real flight scenario because the flight path planning software edits the flight path on the two-dimensional display plane of the electronic device. This results in problems such as low visualization and poor accuracy in environmental representation, which affects the reliability of flight path editing. Summary of the Invention

[0004] In view of this, embodiments of this application provide a route editing method, electronic device, system, and storage medium, aiming to improve the efficiency and reliability of route editing.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] In a first aspect, embodiments of this application provide a flight path editing method applied to a head-mounted device, the method comprising:

[0007] A 3D spatial map of the selected location is displayed on a head-mounted device.

[0008] In response to the interactive operation of the remote controller, the flight path is edited based on the interactive interface in the three-dimensional space map to obtain the flight path of the mobile platform.

[0009] Secondly, embodiments of this application provide a flight path editing method applied to a head-mounted device, the method comprising:

[0010] A map of the selected location is displayed on the screen of the head-mounted device;

[0011] In response to the interactive operation of the remote controller, the flight path is edited on the map based on the interactive interface to obtain the flight path of the mobile platform.

[0012] Thirdly, embodiments of this application provide an electronic device, the electronic device including a head-mounted device, the electronic device further including: a processor and a memory for storing a computer program capable of running on the processor, wherein the processor, when used to run the computer program, is configured to:

[0013] The head-mounted device displays a three-dimensional spatial map of the selected location;

[0014] In response to the interactive operation of the remote controller, the flight path is edited based on the interactive interface in the three-dimensional space map to obtain the flight path of the mobile platform.

[0015] Fourthly, embodiments of this application provide an electronic device, the electronic device including a head-mounted device, the electronic device further including: a processor and a memory for storing a computer program capable of running on the processor, wherein the processor, when used to run the computer program, is configured to:

[0016] A map of the selected location is displayed on the screen of the head-mounted device;

[0017] In response to the interactive operation of the remote controller, the flight path is edited on the map based on the interactive interface to obtain the flight path of the mobile platform.

[0018] Fifthly, embodiments of this application provide a system including the electronic device and remote controller described in the third or fourth aspect of embodiments of this application, wherein the remote controller is communicatively connected to the electronic device.

[0019] Sixthly, embodiments of this application provide a computer storage medium storing a computer program, which, when executed by a processor, implements the steps of the method described in the first or second aspect of embodiments of this application.

[0020] The technical solution provided in this application embodiment displays a three-dimensional spatial map of a selected location on a head-mounted device's display. Responding to interactive operations by the remote controller, flight path editing is performed on the three-dimensional spatial map using an interactive interface to obtain the flight path of the mobile platform. This allows for immersive flight path editing via remote controller interaction on a three-dimensional spatial map, enhancing the intuitiveness of the editing process and significantly improving editing efficiency compared to real-world test flights. Furthermore, the high degree of visualization allows for effective viewing of the relationship between the flight path and buildings or terrain in the three-dimensional spatial map, improving the reliability of flight path editing. Attached Figure Description

[0021] Figure 1 is a flowchart illustrating the route editing method according to an embodiment of this application;

[0022] Figure 2 is a schematic diagram of the interactive identifier in an embodiment of this application;

[0023] Figure 3 is a schematic diagram of the operation prompt window in the state of the remote control touching a waypoint in an application example of this application;

[0024] Figure 4 is a schematic diagram of the operation prompt window in an application example of this application when the remote controller is not touching a waypoint;

[0025] Figure 5 is a schematic diagram of a remote control model and its first operation prompt when the remote control does not touch a waypoint in an application example of this application;

[0026] Figure 6 is a schematic diagram of a remote control model and its third operation prompt when the remote control is not touching a waypoint and the trigger button is pressed in an application example of this application;

[0027] Figure 7 is a schematic diagram of a remote control model and its sixth operation prompt when the remote control touches a waypoint in an application example of this application;

[0028] Figure 8 is a schematic diagram of a remote control touching a waypoint in an application example of this application;

[0029] Figure 9 is a schematic diagram of waypoint movement based on remote control in an application example of this application;

[0030] Figure 10 is a schematic diagram of a three-dimensional spatial map and interactive interface displayed on the display of a head-mounted device in an application example of this application;

[0031] Figure 11 is a schematic diagram of a three-dimensional spatial map and interactive interface displayed on the display of a head-mounted device in another application example of this application;

[0032] Figure 12 is a schematic diagram of the operation prompt window related to the route template in an application example of this application;

[0033] Figures 13A to 13D are schematic diagrams of route editing based on route templates in an application example of this application;

[0034] Figure 14 is a flowchart illustrating the route editing method in an application embodiment of this application;

[0035] Figure 15 is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0036] The present application will now be described in further detail with reference to the accompanying drawings and embodiments.

[0037] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0038] In related technologies, due to the limitations of the two-dimensional display plane of electronic devices, it is difficult to visually edit the flight paths of mobile platforms (e.g., drones). Often, test flights in the real world are required to edit flight paths, impacting editing efficiency. Therefore, this application provides a flight path editing method that introduces a head-mounted device. The head-mounted device's display shows a map, and combined with remote control interaction, online editing of the flight path of a mobile platform is achieved. The map display can be a three-dimensional or two-dimensional spatial map; for example, a three-dimensional spatial map can be displayed, showing a three-dimensional scene, enabling flight path editing in a three-dimensional environment, thus improving the efficiency and reliability of flight path editing.

[0039] As shown in Figure 1, the method for editing the flight route includes:

[0040] Step 101: Display a 3D spatial map of the selected location on the display of the head-mounted device.

[0041] Here, the display of a head-mounted device can be a display device that supports AR (Augmented Reality), VR (Virtual Reality), or MR (Mixed Reality) technologies. AR technology uses computer graphics to simulate and model physical entities, essentially turning the real world into a virtual world. VR technology uses computer graphics and visualization techniques to create virtual objects that don't exist in the physical world and accurately "places" these virtual objects within the physical world, making the virtual world a component of the real world. MR technology establishes an interactive relationship between the virtual and real worlds, creating a hybrid world where virtual and reality interact. In other words, the display of a head-mounted device can be understood as a display device that supports XR (Extended Reality), which refers to an interactive environment combining real and virtual elements, created through computer technology and wearable devices. Extended Reality includes the aforementioned AR, VR, and MR technologies.

[0042] It is understood that the method in this application embodiment, by introducing a head-mounted device for flight path editing, can greatly enrich the display effect during flight path editing and enhance the human-computer interaction experience.

[0043] It should be noted that in other embodiments, the type of map displayed on the head-mounted device's display is not limited, that is, it can be a two-dimensional spatial map or a three-dimensional spatial map.

[0044] Exemplarily, after displaying a three-dimensional spatial map of the selected location on the display of the head-mounted device, the method further includes:

[0045] In response to the first adjustment operation, the current viewing position is adjusted on the three-dimensional spatial map;

[0046] The adjusted 3D spatial map corresponding to the current viewing position is displayed.

[0047] Understandably, users can freely switch their current viewing position within the 3D spatial map displayed on the head-mounted display device's screen based on the first adjustment operation, thereby achieving an immersive 3D flight path editing experience and effectively enhancing the fun of flight path editing.

[0048] For example, the first adjustment operation includes: operation based on a remote control and / or operation based on a head-mounted device. In other words, after wearing the head-mounted device, the user can perform the first adjustment operation based on head movements and / or operations on the remote control, thereby enabling the user to switch between walking, changing viewing angles, etc., in a three-dimensional spatial map. Here, the remote control can be a motion-sensing remote control, for example, it can be based on inertial detection and / or optical detection to recognize, track, quantify, and process the real motion state of the human body, thereby recognizing the user's relevant operations.

[0049] For example, adjusting the current viewing position on the three-dimensional spatial map in response to the first adjustment operation includes:

[0050] When the first adjustment operation is based on a remote control operation, in viewing mode, the position indicated by the remote control on the 3D spatial map is obtained, and the position indicated by the remote control on the 3D spatial map is taken as the current viewing position; or

[0051] When the first adjustment operation is based on the head-mounted device, in viewing mode, the position indicated by the head-tracking center of the head-mounted device on the three-dimensional spatial map is obtained, and the position indicated by the head-tracking center is used as the current viewing position; or

[0052] When the first adjustment operation is based on both remote control operation and head-mounted device operation, the position indicated by the head tracking center of the head-mounted device is obtained, and when the confirmation command sent by the remote control based on the position indicated by the head tracking center is received, the position indicated by the head tracking center is taken as the current viewing position.

[0053] In one application example, the 3D spatial map displayed on the head-mounted device's screen supports a viewing mode and a flight path editing mode. In viewing mode, the user's current viewing position can be adjusted based on the operation of the head-mounted device or the remote control, thereby dynamically switching the user's position and / or viewpoint on the 3D spatial map. In flight path editing mode, flight paths can be edited on the 3D spatial map through an interactive interface based on the remote control's interactive operation. It should be noted that the user's current viewing position can also be switched in flight path editing mode. The switching between the viewing mode and flight path editing mode can be implemented based on a set operation. The head-mounted device's screen displaying the 3D spatial map of the selected location can default to either viewing mode or flight path editing mode; this embodiment does not limit this.

[0054] Understandably, when the first adjustment operation is based on a remote control, in viewing mode, the position indicated by the remote control on the 3D spatial map is obtained, and this position is used as the current viewing position. Alternatively, when the first adjustment operation is based on a head-mounted device, in viewing mode, the position indicated by the head tracking center of the head-mounted device on the 3D spatial map is obtained, and this position is used as the current viewing position. Thus, in viewing mode, users can freely switch their current viewing position, enabling dynamic switching of the user's position and / or perspective on the 3D spatial map. This provides an immersive 3D flight path editing experience and effectively enhances the enjoyment of flight path editing.

[0055] It is understandable that if the display of the 3D spatial map is in the flight path editing mode, the first adjustment operation can be a combination of operation based on the remote control and operation based on the head-mounted device. For example, the head-mounted device can obtain the position indicated by the head tracking center and receive a confirmation command sent by the remote control based on the position indicated by the head tracking center. Then, the position indicated by the head tracking center is taken as the current viewing position, and the user's viewing position can be switched in the flight path editing mode.

[0056] For example, when the first adjustment operation is performed by a head-mounted device, in viewing mode, the current viewing position can be adjusted in real time based on the movement of the head-mounted device to achieve a viewing effect where the user moves on the three-dimensional spatial map. This greatly enhances the intelligence and convenience of real-time adjustment, effectively improving the user's immersive interactive experience of editing flight routes within the three-dimensional spatial map.

[0057] For example, the method further includes:

[0058] In response to the second adjustment operation, the display scale of the three-dimensional spatial map is adjusted.

[0059] Understandably, users can also trigger a second adjustment operation based on the head-mounted device or remote control to adjust the display ratio of the spatial map. For example, the display ratio can be adjusted to a 1:1 display ratio in the real scene, thereby achieving an immersive display effect for the user and making it easier for the user to view the spatial scene from any position in the three-dimensional spatial scene.

[0060] For example, the display based on the head-mounted device displays a three-dimensional spatial map of the selected location, including:

[0061] Three-dimensional map data is obtained based on the selected location, and a three-dimensional spatial map is rendered and displayed based on the three-dimensional map data.

[0062] An interactive interface is overlaid on the three-dimensional spatial map. The interactive interface is used for human-computer interaction on the three-dimensional spatial map, and the interactive interface displays the interactive icon corresponding to the remote control.

[0063] Here, the head-mounted device can send a data acquisition request to the server based on the selected location indicated by the user. The server returns 3D map data to the head-mounted device based on the data acquisition request, and the head-mounted device renders and displays a 3D spatial map based on the received 3D map data.

[0064] For example, a user can use a remote control to select a location on the default 3D spatial map displayed on the head-mounted device's screen and update the map's display. Since the head-mounted device's screen can overlay an interactive interface, including interactive icons, onto the 3D spatial map, the user can perform remote control-related interactive operations based on the interactive icons corresponding to the remote control, thus achieving control during the route editing process.

[0065] Step 102: In response to the interactive operation of the remote controller, the flight path is edited based on the interactive interface in the three-dimensional space map to obtain the flight path of the mobile platform.

[0066] It is understood that in this embodiment, the head-mounted device and the remote controller are communicatively connected, for example, through wireless communication. Thus, after the user wears the head-mounted device, they can input relevant interactive operations based on the remote controller. The head-mounted device can respond to the user's input and perform real-time flight path editing on a 3D spatial map displayed on its screen, based on the interactive interface, to obtain the flight path of the mobile platform. This mobile platform can be a drone or other mobile device with flight capabilities; this embodiment does not limit this to such devices.

[0067] The flight path editing method of this application embodiment can be used to edit flight paths in a three-dimensional spatial map based on the interactive operation of a remote controller, realizing immersive flight path editing, improving the intuitiveness of flight path editing, and effectively improving editing efficiency compared with the test flight method in the real world; in addition, due to the high degree of visualization, the relationship between the flight path and buildings or terrain in the three-dimensional spatial map can be effectively viewed, improving the reliability of flight path editing.

[0068] For example, the interactive identifier in the interactive interface can be a remote control model, and the interactive interface is also used to display corresponding operation prompts on the remote control model.

[0069] In one application example, as shown in Figure 2, the display of the head-mounted device overlays a remote control model on a three-dimensional spatial map. This remote control model reflects the current pose of the remote control, which includes the current movement position and posture. That is, based on this interactive identifier, the user can intuitively know the current position and posture of the remote control in their hand, making it easier for the user to perform relevant operations of the remote control while wearing the head-mounted device, thereby better completing the route editing in the scenario of this application embodiment and improving the user experience.

[0070] Exemplarily, the method further includes:

[0071] Obtain the status of the remote control;

[0072] Based on the state of the remote control, corresponding operation prompts are generated.

[0073] To better guide user operations, enhance the display effect of the interactive interface, and simplify the difficulty of route editing, in this embodiment, the head-mounted device can also acquire the status of the remote controller and generate corresponding operation prompts based on the status of the remote controller. Thus, users can input corresponding interactive operations according to the operation prompts on the remote controller model to complete interactive control related to route editing.

[0074] It is understood that the aforementioned operation prompts are prompts that guide users to generate various interactive operations based on the remote control. For example, they may include operation icons and / or text prompts displayed on the remote control model. This application embodiment does not limit this.

[0075] For example, generating corresponding operation prompts based on the state of the remote control includes:

[0076] If it is determined that the remote controller is in a state of not touching a waypoint, a first type of operation prompt is generated;

[0077] If it is determined that the remote controller is in a state of touching a waypoint, a second type of operation prompt is generated;

[0078] The first type of operation prompts includes one or more of the following: a first operation prompt to move the location, a second operation prompt to rotate the map, a third operation prompt to zoom the map, and a fourth operation prompt to create a new waypoint; the second type of operation prompts includes one or more of the following: a fifth operation prompt to select a waypoint, a sixth operation prompt to adjust the waypoint altitude, and a seventh operation prompt to move the waypoint.

[0079] To better provide prompts to users, the system can be further subdivided based on whether the remote control is in a state of not touching a waypoint. For example, when the remote control is in a state of not touching a waypoint, the first operation prompt to move the location can be displayed. If the remote control is in a state of not touching a waypoint and the trigger button is pressed, the third operation prompt to zoom the map can be displayed. If the remote control is in a state of not touching a waypoint and the orientation is tilted, the second operation prompt to rotate the map can be displayed.

[0080] Understandably, the head-mounted device can generate two types of operation prompts based on the remote controller's status. The first type of operation prompt corresponds to the state where the remote controller has not touched a waypoint, and the second type of operation prompt corresponds to the state where the remote controller has touched a waypoint. For example, in addition to the operation icons displayed on the remote controller model, the operation prompts may also include an operation prompt window overlaid on the 3D spatial map.

[0081] Figure 3 shows a schematic diagram of the operation prompt window in an application example where the remote controller is touching a waypoint. It can be understood that during route editing, the user can generate multiple waypoints in a 3D spatial map based on the remote controller's interactive operations, and then generate a flight route based on the determined waypoints. If the remote controller is in a waypoint-touching state, the operation prompt window can display parameters such as the waypoint number, cruising speed, and waypoint altitude. It can also display parameters such as the total number of waypoints corresponding to the flight route, the total distance, and the estimated flight time. For example, the operation prompt window can also display auxiliary interactive options, such as zooming in, zooming out, and deleting the waypoint on the 3D spatial map. The user can select the corresponding interactive option in the operation prompt window based on the remote controller to achieve interactive control.

[0082] For example, the 3D spatial map and interactive interface displayed on the head-mounted display can move with the movement of the head-mounted device, that is, the 3D spatial map and interactive interface presented to the user are the user's perspective, which helps to enhance the user's interactive experience.

[0083] Figure 4 shows a schematic diagram of the operation prompt window in an application example where the remote controller is not touching a waypoint. If the remote controller is not touching a waypoint, the operation prompt window can display parameters such as the total number of waypoints corresponding to the flight route, the total flight distance, and the estimated flight time. For example, the operation prompt window can also display auxiliary interactive options, such as zooming in and out of the 3D map. Users can select the corresponding interactive options based on the remote controller to achieve interactive control. The operation prompt window can also output text prompts, such as a text prompt indicating that a waypoint has been selected.

[0084] For example, when the remote controller is not touching a waypoint, the first type of operation prompts include, but are not limited to: a first operation prompt to move the location, a second operation prompt to rotate the map, a third operation prompt to zoom the map, and a fourth operation prompt to create a new waypoint. When the remote controller is touching a waypoint, the second type of operation prompts include, but are not limited to: a fifth operation prompt to select a waypoint, a sixth operation prompt to adjust the waypoint altitude, and a seventh operation prompt to move the waypoint.

[0085] In one application example, a remote control is equipped with a trigger button to initiate a specific action. The headset can also differentiate operation prompts based on the state of this trigger button. A specific example is as follows:

[0086] Figure 5 shows a schematic diagram of the remote controller model and its first operation prompt when the remote controller is not touching a waypoint. It can be understood that when the remote controller is not touching a waypoint, the user can control the position of the interactive marker (i.e., the remote controller model) in the 3D spatial map based on the remote controller, for example, tilting up to move forward, tilting down to move backward, tilting left to move left, and tilting right to move right.

[0087] Figure 6 shows a schematic diagram of a remote control model and its third operation prompt when the remote control is not touching a waypoint and the trigger button is pressed. It can be understood that when the remote control is not touching a waypoint and the trigger button is pressed, the user can adjust the display scale of the 3D map using the remote control; for example, tilting the remote control up to zoom in and tilting it down to zoom out.

[0088] Figure 7 shows a schematic diagram of the remote controller model and its sixth operation prompt when the remote controller touches a waypoint. It can be understood that when the remote controller touches a waypoint, the user can adjust the waypoint altitude based on the remote controller's position; for example, after the remote controller touches a waypoint, tilting the controller up increases the waypoint altitude, and tilting it down decreases the waypoint altitude.

[0089] It is understood that the above-mentioned remote control status and corresponding operation prompts are merely illustrative examples. Those skilled in the art can make personalized settings based on individual needs, and this application embodiment does not limit this.

[0090] For example, the step of responding to the interactive operation of the remote controller and editing the flight path in the three-dimensional space map based on the interactive interface to obtain the flight path of the mobile platform includes:

[0091] The interaction between the interaction identifier and the three-dimensional spatial position on the three-dimensional spatial map is obtained, and the waypoint position is determined based on the interaction. The interaction identifier is used to reflect the current pose of the remote controller, and the current pose includes: the current movement position and attitude.

[0092] The flight path of the mobile platform is determined based on the location of the waypoints.

[0093] Understandably, the head-mounted device can respond to the interactive operation of the remote control, obtain the interaction between the interactive identifier and the three-dimensional spatial position on the three-dimensional spatial map under the user's interactive behavior, that is, the interaction between the remote control model in the interactive interface and the three-dimensional spatial position. The remote control model can map the position and posture of the real remote control in the user's hand, thereby realizing an immersive route editing experience in the three-dimensional spatial map, determining the waypoint position in this editing scenario, and generating the flight route of the mobile platform based on the determined waypoint position.

[0094] For example, the interactive operation includes one or more of the following:

[0095] The first interactive action to select a waypoint;

[0096] The second interactive operation for creating a new waypoint;

[0097] The third interactive operation for adjusting waypoint altitude;

[0098] The fourth interactive operation for moving waypoints;

[0099] The fifth interactive operation is rotating the display perspective of the 3D spatial map;

[0100] The sixth interactive operation is to adjust the display scale of the 3D spatial map;

[0101] Adjust the seventh interactive operation of the interactive marker in the 3D spatial map to indicate the current location.

[0102] Understandably, users can dynamically adjust the flight path editing process based on the various interactive operations described above. For example, they can adjust the position of the remote control model in the 3D spatial map using the seventh interactive operation, and can also switch the display view of the 3D spatial map at the current location using the fifth interactive operation. This allows for a 360-degree view of the relationship between the flight path and the terrain, making it easy to intuitively observe whether the flight path will encounter obstacles. Furthermore, the sixth interactive operation allows for proportional zooming in or out of the 3D spatial map, enabling switching between different 3D spaces within the map.

[0103] For example, users can also perform waypoint-related interactive operations on the three-dimensional spatial map. For instance, they can select a waypoint based on the first interactive operation, create a new waypoint based on the second interactive operation, adjust the waypoint altitude based on the third interactive operation, and move the waypoint based on the fourth interactive operation. In this way, the waypoint position can be determined on the three-dimensional spatial map based on the combination of the above interactive operations. This waypoint position is a three-dimensional spatial position. Then, the flight path of the mobile platform can be determined based on the determined waypoint position.

[0104] In one application example, the remote control's interactive operations are defined as follows:

[0105] First interactive operation: After the remote control touches the waypoint, briefly press the trigger button;

[0106] Second interactive operation: Briefly press the trigger button on the remote control to create a new waypoint at the end of the remote control;

[0107] Third interactive operation: After the remote control touches the waypoint, tilting the joystick up will raise the waypoint altitude, and tilting it down will lower the waypoint altitude;

[0108] Fourth interactive operation: After the remote controller touches a waypoint, you can move the waypoint by continuously pressing and holding the trigger button;

[0109] Fifth interactive operation: When the remote controller is not touching a waypoint, press and hold the trigger button and shake the remote controller left and right to rotate the map;

[0110] Sixth interactive operation: When the remote controller is not touching a waypoint, press and hold the trigger button to tilt up or right to zoom in on the map, and press and hold the trigger button to tilt down or left to zoom out on the map;

[0111] Seventh interactive operation: When the remote control is not touching a waypoint, tilt up to move forward, tilt down to move backward, tilt left to move left, and tilt right to move right.

[0112] It is understood that the above interactive operations are merely illustrative examples, and those skilled in the art can make personalized settings based on individual needs. This application does not limit the specifics of these settings.

[0113] For example, the method further includes:

[0114] In response to the third adjustment operation, the current viewing position is moved to the selected waypoint position, and the viewing view corresponding to the waypoint position is displayed.

[0115] Understandably, when users are wearing head-mounted devices and editing flight routes, i.e. in flight route editing mode, they can quickly jump to waypoints based on third-party adjustment operations and display the viewing angle corresponding to the waypoints. This makes it easier for users to edit flight routes from any angle based on the waypoints, thus improving the flight route editing experience.

[0116] For example, the step of responding to the third adjustment operation by moving the current viewing position to the selected waypoint and displaying the viewing angle corresponding to the waypoint position includes:

[0117] The waypoint is selected based on the head tracking center of the head-mounted device or the waypoint is selected based on the remote control. In response to a transfer command, the current viewing position is transferred to the waypoint and the viewing angle corresponding to the waypoint is displayed.

[0118] Understandably, when a head-mounted device displays a 3D spatial map in flight path editing mode, the user can also switch their position and / or viewpoint within the 3D spatial map based on third-party adjustment operations. In one example, if the head-tracking center of the head-mounted device is within a set distance of any waypoint, it can instantly jump to that waypoint location in response to a transfer command (e.g., pressing a setting button on the remote control). After jumping to that waypoint, the user can switch their viewing angle by rotating the head-mounted device, thus achieving a panoramic view of that waypoint location. In another example, if the touch point of the remote control is within a set distance of any waypoint, it can instantly jump to that waypoint location in response to a transfer command (e.g., pressing a setting button on the remote control). In one example, the user's current viewing position instantly jumps to the waypoint location. After jumping to the waypoint location, the user can switch the viewing angle by rotating the head-mounted device, thus achieving a panoramic view of the waypoint location. In another example, assuming the remote controller has a ray positioning function, if the specified ray of the remote controller is within a set distance of any waypoint, it can respond to the generation of a transfer command (e.g., the setting button on the remote controller is pressed) to instantly jump the user's current viewing position to the waypoint location. After jumping to the waypoint location, the user can switch the viewing angle by rotating the head-mounted device, thus achieving a panoramic view of the waypoint location.

[0119] For example, to enhance the user interaction experience, the method further includes:

[0120] In response to the seventh interactive operation, if it is determined that the remote controller is in a state of touching a waypoint, the touched waypoint is highlighted.

[0121] As shown in Figure 8, if the user touches waypoint 3 using the remote control based on the seventh interactive operation, waypoint 3 in the three-dimensional space map will be enlarged to indicate to the user that the waypoint has been touched.

[0122] Exemplarily, the method further includes:

[0123] In response to the fourth interactive operation, reference position points are displayed on the three-dimensional space map to facilitate the movement of the waypoints.

[0124] In one application example, as shown in Figure 9, after the user manipulates the remote control to touch waypoint 3 in the 3D space map, and then holds down the trigger button, for example, after holding down the trigger button for 300 milliseconds, reference position points that facilitate the movement of waypoint 3 can be displayed on the 3D space map. For example, a reference position point can be displayed at a set distance in the 3D space where waypoint 3 is located. In this way, it is convenient for the user to accurately locate when moving waypoints, improve the positioning accuracy of waypoints, and thus improve the accuracy and efficiency of route editing.

[0125] Figures 10 and 11 illustrate the three-dimensional spatial map displayed on the head-mounted device's screen and the interactive interface, respectively. Figure 10 shows the scenario where the remote control touches a waypoint in the three-dimensional spatial map, and Figure 11 shows the scenario where the remote control does not touch a waypoint in the three-dimensional spatial map.

[0126] For example, determining the flight path of the mobile platform based on the determined waypoint locations includes:

[0127] Based on the determined waypoint locations, set the framing direction corresponding to at least one waypoint;

[0128] Based on the determined waypoint locations and the framing direction, a flight path is generated.

[0129] Understandably, users can use a remote control and a head-mounted device to edit flight routes online on a 3D spatial map based on an interactive interface. For example, based on the aforementioned interactive operation, they can determine waypoint locations in 3D space and set the corresponding viewing direction for the determined waypoint locations. Then, based on the flight route generation algorithm, the determined waypoint locations and viewing directions can be connected to form a flight route.

[0130] In one application example, the framing direction of a waypoint can be set by the user through interactive operations on the remote control after selecting the waypoint, or by setting it based on the operation options on the operation prompt window. This application embodiment does not limit this.

[0131] In some embodiments, to improve the efficiency of flight path editing, flight path templates can be introduced into the three-dimensional spatial map, and flight path editing can be performed based on the flight path templates. For example, the step of responding to the interactive operation of the remote controller and editing the flight path in the three-dimensional spatial map based on the interactive interface to obtain the flight path of the mobile platform includes:

[0132] Obtain a route template selected on the interactive interface by the remote controller, the route template including: one or more route segments;

[0133] Determine the starting point of the route template on the three-dimensional spatial map;

[0134] The flight path of the mobile platform is generated based on the starting point of the flight path and the flight path template.

[0135] For example, the head-mounted device can overlay an operation prompt window related to the flight route template on a 3D spatial map based on the status of the remote control. That is, the interactive interface on the 3D spatial map can include the operation prompt window. For example, the display of the head-mounted device can be triggered by a setting button on the remote control to display the operation prompt window related to the flight route template.

[0136] Figure 12 shows a schematic diagram of the operation prompt window related to a flight route template in an application example. This operation prompt window can display interactive options for creating a new flight route segment, allowing the user to select a flight route template. Furthermore, the window can also display parameters such as the total flight distance and estimated flight time corresponding to the flight route. For example, the operation prompt window can also display auxiliary interactive options, such as zooming in and out of the 3D map, deleting the flight route segment, adjusting the size of the flight route segment, and rotating the flight route segment. Users can select the corresponding interactive options in the operation prompt window using a remote control to achieve interactive control.

[0137] In this embodiment, the flight path template includes a first template corresponding to a single camera movement and a second template corresponding to a combination of camera movements. The first template corresponds to a flight path segment for a single camera movement, and the second template corresponds to a flight path package for multiple camera movements. This flight path package can contain at least two flight path segments corresponding to two camera movements. It should be noted that the flight path package is derived from multiple flight path segments, specifically by first connecting the line segments between each flight path segment and then smoothly transitioning them.

[0138] For example, the first template includes, but is not limited to, the following templates: cutaway, fade away, sweep, soar, and circle. The second template can be obtained by combining at least two of the first templates. In this way, the template options for users to edit flight paths can be greatly enriched, thereby achieving the desired flight path editing effect.

[0139] It is understood that the flight path template in this application embodiment is a pre-edited flight path segment. The first template corresponds to the flight path segment of a single camera movement, and the second template is a combination of flight path segments corresponding to combined camera movements. In this way, after determining the starting point of the flight path template in the three-dimensional space map, the camera movement shooting effect of mobile platforms such as drones at specific locations can be achieved based on the flight path corresponding to the flight path template. This can improve the efficiency of flight path editing while meeting the needs of flight path editing.

[0140] For example, the generation of the route based on the route origin and the route template includes:

[0141] The route segment in the route template is adjusted based on the interactive interface, and the route is generated based on the adjusted route template and the selected route starting point.

[0142] In this embodiment, based on a three-dimensional spatial map and the interactive operation of a remote controller, the flight route segments in the flight route template can be adjusted according to the interactive interface, thereby dynamically optimizing the flight route and improving the flight route editing experience.

[0143] For example, the adjustment of the route segment includes one or more of the following:

[0144] Adjust the size of the flight segment;

[0145] Rotate the aforementioned flight path segment;

[0146] Delete the aforementioned route segment.

[0147] Here, adjusting the size of a flight line segment can be understood as zooming in or out on the flight line segment in a 3D spatial map, allowing for a better observation of the distribution of flight line segments in the 3D virtual space; rotating a flight line segment can be understood as changing the direction of the flight line segment in a 3D spatial map, thus allowing for better setting of the flight line segment's orientation; deleting a flight line segment can be understood as allowing the user to decide whether a flight line segment is suitable based on its distribution in the 3D spatial map, and if not, to delete the imported flight line segment.

[0148] Figures 13A to 13D illustrate a schematic diagram of flight path editing based on flight path templates in an application example. Figure 13A shows the interface for creating a new flight path segment in the operation prompt window. Users can select the option to create a new flight path segment using the remote control, entering the interface shown in Figure 13B. This interface displays various preset flight path templates. Users can select the template to add using the remote control. For example, selecting "Snap" as shown in Figure 13C adds the corresponding flight path segment ① to the 3D space map. Users can adjust the starting point of flight path segment ① using the remote control. Following a similar operation, as shown in Figure 13D, users can continue to add corresponding flight path segments ② around and ③ passing over in the 3D space map until the flight path editing is complete, resulting in the flight path.

[0149] It should be noted that in this embodiment of the application, the user adjusts the flight path segment based on the remote control in two ways: the first is to adjust the flight path segment in space based on the remote control; the second is to adjust the flight path segment in the operation prompt window based on the remote control, that is, to adjust the flight path segment by selecting the auxiliary interaction option in the operation prompt window through the remote control.

[0150] In one application example, adjusting a flight path segment in space based on a remote controller can be defined as follows:

[0151] 1) Select route segment: After touching the route segment, briefly press the trigger button;

[0152] 2) Zoom in / out of the flight path: After touching the flight path, tilt up and right to zoom in, and tilt down and right to zoom out;

[0153] 3) Rotate the route: After touching the route segment, press the trigger button briefly, and then rotate the remote control. The route will rotate according to the rotation angle of the remote control.

[0154] 4) Delete route segment: After selecting the route segment, quickly throw the remote control.

[0155] Exemplarily, the method further includes:

[0156] Receive a flight path preview video generated by the server based on the flight path;

[0157] The flight route preview video is displayed.

[0158] In this embodiment, the user completes flight path editing in a 3D spatial map based on the interactive operation of the remote control. After obtaining the flight path of the mobile platform, the user can also receive a flight path preview video generated by the server based on the flight path. For example, the user can send a request to the server to obtain the flight path preview video to the head-mounted device. The server generates the flight path preview video based on the request and sends it to the head-mounted device. The head-mounted device then displays the flight path preview video to the user. In this way, the user can immediately view the flight path preview video after completing the flight path editing, and thus further determine whether the flight path editing meets the expected effect.

[0159] It should be noted that the scenery in the flight route preview video is generated by the server based on map modeling data, and is not actually filmed. Since the map modeling data has simulated the height, size, etc. of real buildings, the flight route preview video can simulate the aerial photography effect of actual flight, making it easier for users to judge whether the flight route design is reasonable.

[0160] It should be noted that the embodiments of this application do not limit the type of map displayed on the display of the head-mounted device, that is, it can be a two-dimensional spatial map or a three-dimensional spatial map. If the map type is a two-dimensional spatial map, it may affect the user's visual experience. The relevant methods for flight route editing can be referred to the foregoing embodiments, and will not be repeated here.

[0161] The present application will now be described in further detail with reference to an application embodiment.

[0162] In this application embodiment, as shown in Figure 14, the route editing method is applied to a system including a cloud server, an XR head-mounted display (i.e., the aforementioned head-mounted device), and a remote controller. The cloud server can be understood as a map data server, used to send map data in response to location requests from the XR head-mounted display. The XR head-mounted display is used to display a three-dimensional spatial map and its interactive interface. The remote controller serves as the interactive operation tool for the user wearing the head-mounted display. The remote controller is wirelessly connected to the XR head-mounted display and can send interactive operations to it. The XR head-mounted display can respond to the interactive operations of the remote controller by displaying the interactive interface on the three-dimensional spatial map, facilitating route editing by the user based on the interactive interface. The specific control process includes the following steps:

[0163] Step 1401: Request a 3D map of the selected location.

[0164] Here, users can request a 3D map of a selected location based on an XR head-mounted display.

[0165] Step 1402: Search for the corresponding 3D map and send the map data to the XR head-mounted display.

[0166] Here, the cloud server searches for a 3D map of the selected location and sends the map data to the XR head-mounted display.

[0167] Step 1403: Display the 3D map and the corresponding UI interface.

[0168] Here, after receiving map data, the XR head-mounted display renders and displays a three-dimensional spatial map and overlays a UI interface on the three-dimensional spatial map. The UI interface includes a remote control model mapped to the three-dimensional spatial map.

[0169] Step 1404: The remote control moves its position in space.

[0170] Here, users can control the remote control to move its position in space.

[0171] Step 1405: The infrared (IR) sensor tracks the position and orientation of the remote control.

[0172] Understandably, the infrared sensors on the head-mounted display can track the position and orientation of the remote control, facilitating dynamic adjustment of the remote control model displayed in the 3D spatial map, so that the remote control model displayed in the 3D spatial map can map the position and posture of the remote control in the user's hand.

[0173] Step 1406: On the spatial interface, display the dynamic model of the handle according to the tracking position and direction.

[0174] Here, the head-mounted display dynamically displays a remote control model based on the tracked position and direction, so that the remote control model displayed in the 3D space map can map the position and posture of the remote control in the user's hand.

[0175] Step 1407: Send control commands.

[0176] Understandably, users can generate and send control commands to the head-mounted display via the remote control based on the UI interface.

[0177] Step 1408: Receive control command.

[0178] Step 1409: The head-mounted display determines whether it has detected interactive events in the 3D spatial map.

[0179] Here, an interactive event is an interactive operation generated by the user based on the remote control.

[0180] Step 1410: Respond to the corresponding interaction, and the spatial interface changes accordingly.

[0181] Understandably, based on the images displayed by the remote controller and head-mounted display, users can edit flight routes online in a three-dimensional virtual space. Moreover, the interactive operation based on the remote controller makes editing quick and simple, which can effectively improve editing efficiency compared to real-world test flights. In addition, due to the high degree of visualization, the relationship between the flight route and buildings or terrain in the three-dimensional space map can be effectively viewed, improving the reliability of flight route editing.

[0182] To implement the methods of the embodiments of this application, an electronic device is also provided. FIG15 only shows an exemplary structure of the electronic device and not the entire structure; some or all of the structures shown in FIG15 can be implemented as needed.

[0183] As shown in Figure 15, the electronic device 1500 provided in this embodiment includes at least one processor 1501, a memory 1502, a user interface 1503, and at least one network interface 1504. The various components in the electronic device 1500 are coupled together via a bus system 1505. It can be understood that the bus system 1505 is used to implement communication between these components. In addition to a data bus, the bus system 1505 also includes a power bus, a control bus, and a status signal bus. However, for clarity, all buses are labeled as bus system 1505 in Figure 15.

[0184] The user interface 1503 may include a monitor, keyboard, mouse, trackball, click wheel, buttons, touchpad, or touch screen.

[0185] The memory 1502 in this embodiment is used to store various types of data to support the operation of the electronic device. Examples of such data include any computer program used to operate on the electronic device.

[0186] The route editing method disclosed in this application can be applied to or implemented by the processor 1501. The processor 1501 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the route editing method can be completed by the integrated logic circuitry in the hardware of the processor 1501 or by instructions in software form. The processor 1501 can be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 1501 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software modules can be located in a storage medium, specifically memory 1502. The processor 1501 reads information from memory 1502 and, in conjunction with its hardware, completes the steps of the route editing method provided in the embodiments of this application.

[0187] In an exemplary embodiment, the electronic device may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned method.

[0188] For example, the electronic device in this application embodiment may include a head-mounted device, wherein the processor may be a processor built into the head-mounted device.

[0189] For example, the processor is configured as follows:

[0190] The head-mounted device displays a three-dimensional spatial map of the selected location;

[0191] In response to the interactive operation of the remote controller, the flight path is edited based on the interactive interface in the three-dimensional space map to obtain the flight path of the mobile platform.

[0192] For example, the processor is further configured to:

[0193] In response to the first adjustment operation, the current viewing position is adjusted on the three-dimensional spatial map;

[0194] The adjusted 3D spatial map corresponding to the current viewing position is displayed.

[0195] For example, the first adjustment operation includes: operation based on a remote control and / or operation based on a head-mounted device.

[0196] For example, the processor is configured to:

[0197] When the first adjustment operation is based on a remote control operation, in viewing mode, the position indicated by the remote control on the 3D spatial map is obtained, and the position indicated by the remote control on the 3D spatial map is taken as the current viewing position; or

[0198] When the first adjustment operation is based on the head-mounted device, in viewing mode, the position indicated by the head-tracking center of the head-mounted device on the three-dimensional spatial map is obtained, and the position indicated by the head-tracking center is used as the current viewing position; or

[0199] When the first adjustment operation is based on both remote control operation and head-mounted device operation, the position indicated by the head tracking center of the head-mounted device is obtained, and when the confirmation command sent by the remote control based on the position indicated by the head tracking center is received, the position indicated by the head tracking center is taken as the current viewing position.

[0200] For example, the processor is further configured to:

[0201] In response to the second adjustment operation, the display scale of the three-dimensional spatial map is adjusted.

[0202] For example, the processor is configured to:

[0203] Three-dimensional map data is obtained based on the selected location, and a three-dimensional spatial map is rendered and displayed based on the three-dimensional map data.

[0204] An interactive interface is overlaid on the three-dimensional spatial map. The interactive interface is used for human-computer interaction on the three-dimensional spatial map, and the interactive interface displays the interactive icon corresponding to the remote control.

[0205] For example, the processor is configured as follows:

[0206] Three-dimensional map data is obtained based on the selected location, and a three-dimensional spatial map is rendered and displayed based on the three-dimensional map data.

[0207] The interactive interface is overlaid on the three-dimensional spatial map. The interactive interface is used for human-computer interaction on the three-dimensional spatial map, and the interactive interface displays the interactive icon corresponding to the remote control.

[0208] For example, the interaction identifier is a remote control model, and the interaction interface is also used to display corresponding operation prompts on the remote control model.

[0209] For example, the processor is further configured to:

[0210] Obtain the status of the remote control;

[0211] Based on the state of the remote control, corresponding operation prompts are generated.

[0212] For example, the processor is configured to:

[0213] If it is determined that the remote controller is in a state of not touching a waypoint, a first type of operation prompt is generated;

[0214] If it is determined that the remote controller is in a state of touching a waypoint, a second type of operation prompt is generated;

[0215] The first type of operation prompts includes one or more of the following: a first operation prompt to move the location, a second operation prompt to rotate the map, a third operation prompt to zoom the map, and a fourth operation prompt to create a new waypoint; the second type of operation prompts includes one or more of the following: a fifth operation prompt to select a waypoint, a sixth operation prompt to adjust the waypoint altitude, and a seventh operation prompt to move the waypoint.

[0216] For example, the processor is configured to:

[0217] The interaction between the interaction identifier and the three-dimensional spatial position on the three-dimensional spatial map is obtained, and the waypoint position is determined based on the interaction. The interaction identifier is used to reflect the current pose of the remote controller, and the current pose includes: the current movement position and attitude.

[0218] The flight path of the mobile platform is determined based on the location of the waypoints.

[0219] For example, the processor is further configured to:

[0220] In response to the third adjustment operation, the current viewing position is moved to the selected waypoint position, and the viewing view corresponding to the waypoint position is displayed.

[0221] For example, the processor is configured to:

[0222] The waypoint is selected based on the head tracking center of the head-mounted device or the waypoint is selected based on the remote control. In response to a transfer command, the current viewing position is transferred to the waypoint and the viewing angle corresponding to the waypoint is displayed.

[0223] For example, the interactive operation includes one or more of the following:

[0224] The first interactive action to select a waypoint;

[0225] The second interactive operation for creating a new waypoint;

[0226] The third interactive operation for adjusting waypoint altitude;

[0227] The fourth interactive operation for moving waypoints;

[0228] The fifth interactive operation is rotating the display perspective of the 3D spatial map;

[0229] The sixth interactive operation is to adjust the display scale of the 3D spatial map;

[0230] Adjust the seventh interactive operation of the interactive marker in the 3D spatial map to indicate the current location.

[0231] For example, the processor is further configured to:

[0232] In response to the seventh interactive operation, if it is determined that the remote controller is in a state of touching a waypoint, the touched waypoint is highlighted.

[0233] For example, the processor is further configured to:

[0234] In response to the fourth interactive operation, reference position points are displayed on the three-dimensional space map to facilitate the movement of the waypoints.

[0235] For example, the processor is configured to:

[0236] In response to the interactive operation of the remote controller, the waypoint position is determined in the three-dimensional space map based on the interactive interface, and the waypoint position is a three-dimensional space position;

[0237] The flight path of the mobile platform is determined based on the location of the waypoints.

[0238] For example, the processor is configured to:

[0239] Obtain a route template selected on the interactive interface by the remote controller, the route template including: one or more route segments;

[0240] Determine the starting point of the route template on the three-dimensional spatial map;

[0241] The flight path of the mobile platform is generated based on the starting point of the flight path and the flight path template.

[0242] For example, the processor is configured to:

[0243] The route segment in the route template is adjusted based on the interactive interface, and the route is generated based on the adjusted route template and the selected route starting point.

[0244] For example, the adjustment of the route segment includes one or more of the following:

[0245] Adjust the size of the flight segment;

[0246] Rotate the aforementioned flight path segment;

[0247] Delete the aforementioned route segment.

[0248] For example, the processor is configured to:

[0249] Based on the determined waypoint locations, set the framing direction corresponding to at least one waypoint;

[0250] Based on the determined waypoint locations and the framing direction, a flight path is generated.

[0251] For example, the processor is further configured to:

[0252] Receive a flight path preview video generated by the server based on the flight path;

[0253] The flight route preview video is displayed.

[0254] It should be noted that in other embodiments, the type of map displayed on the head-mounted device's display is not limited; it can be a two-dimensional or three-dimensional spatial map. If the map type is a two-dimensional spatial map, it may affect the user's visual experience. The route editing process of the electronic device can be referred to the aforementioned embodiments of the electronic device, and will not be repeated here.

[0255] It is understood that memory 1502 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), Sync Link Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memories described in the embodiments of this application are intended to include, but are not limited to, these and any other suitable types of memories.

[0256] In an exemplary embodiment, this application also provides a system including the electronic device and remote controller described in this application embodiment, wherein the remote controller is communicatively connected to the electronic device. The remote controller can be a motion-sensing remote controller, thus enabling the route editing method of this application embodiment to be implemented based on the cooperation between the remote controller and the electronic device.

[0257] In an exemplary embodiment, this application also provides a computer storage medium, specifically a computer-readable storage medium, such as a memory 1502 storing a computer program, which can be executed by a processor 1501 of an electronic device to complete the steps described in the method of this application embodiment. The computer-readable storage medium can be a ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM, etc.

[0258] In an exemplary embodiment, this application also provides a computer program product, including a computer program that can be executed by a processor 1501 of an electronic device 1500 to perform the steps described in the method of this application embodiment.

[0259] It should be noted that terms such as "first" and "second" are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0260] Furthermore, the technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.

[0261] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for editing flight routes, characterized in that, Applied to head-mounted devices, the method includes: The head-mounted device displays a three-dimensional spatial map of the selected location; In response to the interactive operation of the remote controller, the flight path is edited based on the interactive interface in the three-dimensional space map to obtain the flight path of the mobile platform.

2. The method according to claim 1, characterized in that, After the display of the selected location on the head-mounted device shows a three-dimensional spatial map, the method further includes: In response to the first adjustment operation, the current viewing position is adjusted on the three-dimensional spatial map; The adjusted 3D spatial map corresponding to the current viewing position is displayed.

3. The method according to claim 2, characterized in that, The first adjustment operation includes: operation based on a remote control and / or operation based on a head-mounted device.

4. The method according to claim 3, characterized in that, The adjustment of the current viewing position on the three-dimensional spatial map in response to the first adjustment operation includes: When the first adjustment operation is based on a remote control operation, in viewing mode, the position indicated by the remote control on the 3D spatial map is obtained, and the position indicated by the remote control on the 3D spatial map is taken as the current viewing position; or When the first adjustment operation is based on the head-mounted device, in viewing mode, the position indicated by the head-tracking center of the head-mounted device on the three-dimensional spatial map is obtained, and the position indicated by the head-tracking center is used as the current viewing position; or When the first adjustment operation is based on both remote control operation and head-mounted device operation, the position indicated by the head tracking center of the head-mounted device is obtained, and when the confirmation command sent by the remote control based on the position indicated by the head tracking center is received, the position indicated by the head tracking center is taken as the current viewing position.

5. The method according to claim 1, characterized in that, The method further includes: In response to the second adjustment operation, the display scale of the three-dimensional spatial map is adjusted.

6. The method according to claim 1, characterized in that, The display based on the head-mounted device shows a three-dimensional spatial map of the selected location, including: Three-dimensional map data is obtained based on the selected location, and a three-dimensional spatial map is rendered and displayed based on the three-dimensional map data. An interactive interface is overlaid on the three-dimensional spatial map. The interactive interface is used for human-computer interaction on the three-dimensional spatial map, and the interactive interface displays the interactive icon corresponding to the remote control.

7. The method according to claim 6, characterized in that, The interactive identifier is a remote control model, and the interactive interface is also used to display corresponding operation prompts on the remote control model.

8. The method according to claim 7, characterized in that, The method further includes: Obtain the status of the remote control; Based on the state of the remote control, corresponding operation prompts are generated.

9. The method according to claim 8, characterized in that, The step of generating corresponding operation prompts based on the state of the remote control includes: If it is determined that the remote controller is in a state of not touching a waypoint, a first type of operation prompt is generated; If it is determined that the remote controller is in a state of touching a waypoint, a second type of operation prompt is generated; The first type of operation prompts includes one or more of the following: a first operation prompt to move the location, a second operation prompt to rotate the map, a third operation prompt to zoom the map, and a fourth operation prompt to create a new waypoint; the second type of operation prompts includes one or more of the following: a fifth operation prompt to select a waypoint, a sixth operation prompt to adjust the waypoint altitude, and a seventh operation prompt to move the waypoint.

10. The method according to claim 6, characterized in that, The interaction with the remote controller, in response to the interactive operation, involves editing the flight path on the three-dimensional spatial map based on the interactive interface to obtain the flight path of the mobile platform, including: The interaction between the interaction identifier and the three-dimensional spatial position on the three-dimensional spatial map is obtained, and the waypoint position is determined based on the interaction. The interaction identifier is used to reflect the current pose of the remote controller, and the current pose includes: the current movement position and attitude. The flight path of the mobile platform is determined based on the location of the waypoints.

11. The method according to claim 6, characterized in that, The method further includes: In response to the third adjustment operation, the current viewing position is moved to the selected waypoint position, and the viewing view corresponding to the waypoint position is displayed.

12. The method according to claim 11, characterized in that, The response to the third adjustment operation, moving the current viewing position to the selected waypoint and displaying the viewing angle corresponding to the waypoint position, includes: The waypoint is selected based on the head tracking center of the head-mounted device or the waypoint is selected based on the remote control. In response to a transfer command, the current viewing position is transferred to the waypoint and the viewing angle corresponding to the waypoint is displayed.

13. The method according to claim 1 or 6, characterized in that, The interactive operation includes one or more of the following: The first interactive action to select a waypoint; The second interactive operation for creating a new waypoint; The third interactive operation for adjusting waypoint altitude; The fourth interactive operation for moving waypoints; The fifth interactive operation is rotating the display perspective of the 3D spatial map; The sixth interactive operation is to adjust the display scale of the 3D spatial map; Adjust the seventh interactive operation of the interactive marker in the 3D spatial map to indicate the current location.

14. The method according to claim 13, characterized in that, The method further includes: In response to the seventh interactive operation, if it is determined that the remote controller is in a state of touching a waypoint, the touched waypoint is highlighted.

15. The method according to claim 13, characterized in that, The method further includes: In response to the fourth interactive operation, reference position points are displayed on the three-dimensional space map to facilitate the movement of the waypoints.

16. The method according to claim 1, characterized in that, The step of responding to the interactive operation of the remote controller by editing the flight path on the three-dimensional space map based on the interactive interface to obtain the flight path of the mobile platform includes: In response to the interactive operation of the remote controller, the waypoint position is determined in the three-dimensional space map based on the interactive interface, and the waypoint position is a three-dimensional space position; The flight path of the mobile platform is determined based on the location of the waypoints.

17. The method according to claim 1, characterized in that, The step of responding to the interactive operation of the remote controller by editing the flight path on the three-dimensional space map based on the interactive interface to obtain the flight path of the mobile platform includes: Obtain a route template selected on the interactive interface based on the remote controller, the route template including: one or more route segments; Determine the starting point of the route template on the three-dimensional spatial map; The flight path of the mobile platform is generated based on the starting point of the flight path and the flight path template.

18. The method according to claim 17, characterized in that, The process of generating a route based on the route origin and the route template includes: The route segment in the route template is adjusted based on the interactive interface, and the route is generated based on the adjusted route template and the selected route starting point.

19. The method according to claim 18, characterized in that, The adjustment of the route segment includes one or more of the following: Adjust the size of the flight path segment; Rotate the aforementioned flight path segment; Delete the aforementioned route segment.

20. The method according to claim 10 or 16, characterized in that, Determining the flight path of the mobile platform based on the determined waypoint locations includes: Based on the determined waypoint locations, set the framing direction corresponding to at least one waypoint; Based on the determined waypoint locations and the framing direction, a flight path is generated.

21. The method according to claim 1, characterized in that, The method further includes: Receive a flight path preview video generated by the server based on the flight path; Displaying a preview video of the described flight route.

22. A method for editing flight routes, characterized in that, Applied to head-mounted devices, the method includes: A map of the selected location is displayed on the screen of the head-mounted device; In response to the interactive operation of the remote controller, the flight path is edited on the map based on the interactive interface to obtain the flight path of the mobile platform.

23. The method according to claim 22, characterized in that, The display based on the head-mounted device shows a map of the selected location, including: Obtain map data based on the selected location, and render and display the map based on the map data; An interactive interface is overlaid on the map. The interactive interface is used for human-computer interaction on the map, and the interactive interface displays the interactive icon corresponding to the remote control.

24. The method according to claim 23, characterized in that, The interactive identifier is a remote control model, and the interactive interface is also used to display corresponding operation prompts on the remote control model.

25. The method according to claim 24, characterized in that, The method further includes: Obtain the status of the remote control; Based on the state of the remote control, corresponding operation prompts are generated.

26. The method according to claim 25, characterized in that, The step of generating corresponding operation prompts based on the state of the remote control includes: If it is determined that the remote controller is in a state of not touching a waypoint, a first type of operation prompt is generated; If it is determined that the remote controller is in a state of touching a waypoint, a second type of operation prompt is generated; The first type of operation prompts includes one or more of the following: a first operation prompt to move the location, a second operation prompt to rotate the map, a third operation prompt to zoom the map, and a fourth operation prompt to create a new waypoint; the second type of operation prompts includes one or more of the following: a fifth operation prompt to select a waypoint, a sixth operation prompt to adjust the waypoint altitude, and a seventh operation prompt to move the waypoint.

27. The method according to claim 23, characterized in that, The interaction with the remote controller, in response to the interactive operation, involves editing the flight path on the map based on the interactive interface to obtain the flight path of the mobile platform, including: The interaction between the interaction identifier and the spatial location on the map is obtained, and the waypoint location is determined based on the interaction. The interaction identifier is used to reflect the current pose of the remote controller, and the current pose includes: the current movement position and attitude. The flight path of the mobile platform is determined based on the location of the waypoints.

28. The method according to claim 22 or 23, characterized in that, The interactive operation includes one or more of the following: The first interactive action to select a waypoint; The second interactive operation for creating a new waypoint; The third interactive operation for adjusting waypoint altitude; The fourth interactive operation for moving waypoints; The fifth interactive operation is rotating the map's display perspective; The sixth interactive operation is to adjust the map's display scale; Adjust the seventh interactive action of the current location of the interactive marker on the map.

29. The method according to claim 28, characterized in that, The method further includes: In response to the seventh interactive operation, if it is determined that the remote controller is in a state of touching a waypoint, the touched waypoint is highlighted.

30. The method according to claim 28, characterized in that, The method further includes: In response to the fourth interactive operation, a reference location point is displayed on the map to facilitate movement of the waypoint.

31. The method according to claim 22, characterized in that, The step of responding to the interactive operation of the remote controller by editing the flight path on the map based on the interactive interface to obtain the flight path of the mobile platform includes: In response to the interactive operation of the remote controller, the waypoint position is determined on the map based on the interactive interface; The flight path of the mobile platform is determined based on the location of the waypoints.

32. The method according to claim 22, characterized in that, The step of responding to the interactive operation of the remote controller by editing the flight path on the map based on the interactive interface to obtain the flight path of the mobile platform includes: Obtain a route template selected on the interactive interface based on the remote controller, the route template including: one or more route segments; Determine the starting point of the route template on the map; The flight path of the mobile platform is generated based on the starting point of the flight path and the flight path template.

33. The method according to claim 32, characterized in that, The process of generating a route based on the route origin and the route template includes: The route segment in the route template is adjusted based on the interactive interface, and the route is generated based on the adjusted route template and the selected route starting point.

34. The method according to claim 33, characterized in that, The adjustment of the route segment includes one or more of the following: Adjust the size of the flight path segment; Rotate the aforementioned flight path segment; Delete the aforementioned route segment.

35. The method according to claim 27 or 31, characterized in that, Determining the flight path of the mobile platform based on the determined waypoint locations includes: Based on the determined waypoint locations, set the framing direction corresponding to at least one waypoint; Based on the determined waypoint locations and the framing direction, a flight path is generated.

36. The method according to claim 22, characterized in that, The method further includes: Receive a flight path preview video generated by the server based on the flight path; Displaying a preview video of the described flight route.

37. An electronic device, characterized in that, The electronic device includes a head-mounted device, and further includes a processor and a memory for storing a computer program capable of running on the processor, wherein the processor, when running the computer program, is configured to: The head-mounted device displays a three-dimensional spatial map of the selected location; In response to the interactive operation of the remote controller, the flight path is edited based on the interactive interface in the three-dimensional space map to obtain the flight path of the mobile platform.

38. The electronic device according to claim 37, characterized in that, The processor is also configured to: In response to the first adjustment operation, the current viewing position is adjusted on the three-dimensional spatial map; The adjusted 3D spatial map corresponding to the current viewing position is displayed.

39. The electronic device according to claim 38, characterized in that, The first adjustment operation includes: operation based on a remote control and / or operation based on a head-mounted device.

40. The electronic device according to claim 39, characterized in that, The processor is configured to: When the first adjustment operation is based on the remote control, in viewing mode, the position indicated by the remote control on the three-dimensional space map is obtained, and the position indicated by the remote control on the three-dimensional space map is taken as the current viewing position; or When the first adjustment operation is based on the operation of the head-mounted device, in viewing mode, the position indicated by the head tracking center of the head-mounted device on the three-dimensional space map is obtained, and the position indicated by the head tracking center is used as the current viewing position; or When the first adjustment operation is based on both remote control operation and head-mounted device operation, the position indicated by the head tracking center of the head-mounted device is obtained, and when the confirmation command sent by the remote control based on the position indicated by the head tracking center is received, the position indicated by the head tracking center is taken as the current viewing position.

41. The electronic device according to claim 37, characterized in that, The processor is also configured to: In response to the second adjustment operation, the display scale of the three-dimensional spatial map is adjusted.

42. The electronic device according to claim 37, characterized in that, The processor is configured to: Three-dimensional map data is obtained based on the selected location, and a three-dimensional spatial map is rendered and displayed based on the three-dimensional map data. An interactive interface is overlaid on the three-dimensional spatial map. The interactive interface is used for human-computer interaction on the three-dimensional spatial map, and the interactive interface displays the interactive icon corresponding to the remote control.

43. The electronic device according to claim 42, characterized in that, The interactive identifier is a remote control model, and the interactive interface is also used to display corresponding operation prompts on the remote control model.

44. The electronic device according to claim 43, characterized in that, The processor is also configured to: Obtain the status of the remote control; Based on the state of the remote control, corresponding operation prompts are generated.

45. The electronic device according to claim 44, characterized in that, The processor is configured to: If it is determined that the remote controller is in a state of not touching a waypoint, a first type of operation prompt is generated; If it is determined that the remote controller is in a state of touching a waypoint, a second type of operation prompt is generated; The first type of operation prompts includes one or more of the following: a first operation prompt to move the location, a second operation prompt to rotate the map, a third operation prompt to zoom the map, and a fourth operation prompt to create a new waypoint; the second type of operation prompts includes one or more of the following: a fifth operation prompt to select a waypoint, a sixth operation prompt to adjust the waypoint altitude, and a seventh operation prompt to move the waypoint.

46. ​​The electronic device according to claim 42, characterized in that, The processor is configured to: The interaction between the interaction identifier and the three-dimensional spatial position on the three-dimensional spatial map is obtained, and the waypoint position is determined based on the interaction. The interaction identifier is used to reflect the current pose of the remote controller, and the current pose includes: the current movement position and attitude. The flight path of the mobile platform is determined based on the location of the waypoints.

47. The electronic device according to claim 42, characterized in that, The processor is also configured to: In response to the third adjustment operation, the current viewing position is moved to the selected waypoint position, and the viewing view corresponding to the waypoint position is displayed.

48. The electronic device according to claim 47, characterized in that, The processor is configured to: The waypoint is selected based on the head tracking center of the head-mounted device or the waypoint is selected based on the remote control. In response to a transfer command, the current viewing position is transferred to the waypoint and the viewing angle corresponding to the waypoint is displayed.

49. The electronic device according to claim 37 or 42, characterized in that, The interactive operation includes one or more of the following: The first interactive action to select a waypoint; The second interactive operation for creating a new waypoint; The third interactive operation for adjusting waypoint altitude; The fourth interactive operation for moving waypoints; The fifth interactive operation is rotating the display perspective of the 3D spatial map; The sixth interactive operation is to adjust the display scale of the 3D spatial map; Adjust the seventh interactive operation of the interactive marker in the 3D spatial map to indicate the current location.

50. The electronic device according to claim 49, characterized in that, The processor is also configured to: In response to the seventh interactive operation, if it is determined that the remote controller is in a state of touching a waypoint, the touched waypoint is highlighted.

51. The electronic device according to claim 49, characterized in that, The processor is also configured to: In response to the fourth interactive operation, reference position points are displayed on the three-dimensional space map to facilitate the movement of the waypoints.

52. The electronic device according to claim 37, characterized in that, The processor is configured to: In response to the interactive operation of the remote controller, the waypoint position is determined in the three-dimensional space map based on the interactive interface, and the waypoint position is a three-dimensional space position; The flight path of the mobile platform is determined based on the location of the waypoints.

53. The electronic device according to claim 37, characterized in that, The processor is configured to: Obtain a route template selected on the interactive interface based on the remote controller, the route template including: one or more route segments; Determine the starting point of the route template on the three-dimensional spatial map; The flight path of the mobile platform is generated based on the starting point of the flight path and the flight path template.

54. The electronic device according to claim 53, characterized in that, The processor is configured to: The route segment in the route template is adjusted based on the interactive interface, and the route is generated based on the adjusted route template and the selected route starting point.

55. The electronic device according to claim 54, characterized in that, The adjustment of the route segment includes one or more of the following: Adjust the size of the flight path segment; Rotate the aforementioned flight path segment; Delete the aforementioned route segment.

56. The electronic device according to claim 46 or 52, characterized in that, The processor is configured to: Based on the determined waypoint locations, set the framing direction corresponding to at least one waypoint; Based on the determined waypoint locations and the framing direction, a flight path is generated.

57. The electronic device according to claim 37, characterized in that, The processor is also configured to: Receive a flight path preview video generated by the server based on the flight path; Displaying a preview video of the described flight route.

58. An electronic device, characterized in that, The electronic device includes a head-mounted device, and further includes a processor and a memory for storing a computer program capable of running on the processor, wherein the processor, when running the computer program, is configured to: A map of the selected location is displayed on the screen of the head-mounted device; In response to the interactive operation of the remote controller, the flight path is edited on the map based on the interactive interface to obtain the flight path of the mobile platform.

59. The electronic device according to claim 58, characterized in that, The processor is configured to: Obtain map data based on the selected location, and render and display the map based on the map data; An interactive interface is overlaid on the map. The interactive interface is used for human-computer interaction on the map, and the interactive interface displays the interactive icon corresponding to the remote control.

60. The electronic device according to claim 59, characterized in that, The interactive identifier is a remote control model, and the interactive interface is also used to display corresponding operation prompts on the remote control model.

61. The electronic device according to claim 60, characterized in that, The processor is also configured to: Obtain the status of the remote control; Based on the state of the remote control, corresponding operation prompts are generated.

62. The electronic device according to claim 61, characterized in that, The processor is configured to: If it is determined that the remote controller is in a state of not touching a waypoint, a first type of operation prompt is generated; If it is determined that the remote controller is in a state of touching a waypoint, a second type of operation prompt is generated; The first type of operation prompts includes one or more of the following: a first operation prompt to move the location, a second operation prompt to rotate the map, a third operation prompt to zoom the map, and a fourth operation prompt to create a new waypoint; the second type of operation prompts includes one or more of the following: a fifth operation prompt to select a waypoint, a sixth operation prompt to adjust the waypoint altitude, and a seventh operation prompt to move the waypoint.

63. The electronic device according to claim 59, characterized in that, The processor is configured to: The interaction between the interaction identifier and the spatial location on the map is obtained, and the waypoint location is determined based on the interaction. The interaction identifier is used to reflect the current pose of the remote controller, and the current pose includes: the current movement position and attitude. The flight path of the mobile platform is determined based on the location of the waypoints.

64. The electronic device according to claim 58 or 59, characterized in that, The interactive operation includes one or more of the following: The first interactive action to select a waypoint; The second interactive operation for creating a new waypoint; The third interactive operation for adjusting waypoint altitude; The fourth interactive operation for moving waypoints; The fifth interactive operation is rotating the map's display perspective; The sixth interactive operation is to adjust the map's display scale; Adjust the seventh interactive action of the current location of the interactive marker on the map.

65. The electronic device according to claim 64, characterized in that, The processor is also configured to: In response to the seventh interactive operation, if it is determined that the remote controller is in a state of touching a waypoint, the touched waypoint is highlighted.

66. The electronic device according to claim 64, characterized in that, The processor is also configured to: In response to the fourth interactive operation, a reference location point is displayed on the map to facilitate movement of the waypoint.

67. The electronic device according to claim 58, characterized in that, The processor is configured to: In response to the interactive operation of the remote controller, the waypoint position is determined on the map based on the interactive interface; The flight path of the mobile platform is determined based on the location of the waypoints.

68. The electronic device according to claim 58, characterized in that, The processor is configured to: Obtain a route template selected on the interactive interface based on the remote controller, the route template including: one or more route segments; Determine the starting point of the route template on the map; The flight path of the mobile platform is generated based on the starting point of the flight path and the flight path template.

69. The electronic device according to claim 68, characterized in that, The processor is configured to: The route segment in the route template is adjusted based on the interactive interface, and the route is generated based on the adjusted route template and the selected route starting point.

70. The electronic device according to claim 69, characterized in that, The adjustment of the route segment includes one or more of the following: Adjust the size of the flight path segment; Rotate the aforementioned flight path segment; Delete the aforementioned route segment.

71. The electronic device according to claim 63 or 67, characterized in that, The processor is configured to: Based on the determined waypoint locations, set the framing direction corresponding to at least one waypoint; Based on the determined waypoint locations and the framing direction, a flight path is generated.

72. The electronic device according to claim 58, characterized in that, The processor is also configured to: Receive a flight path preview video generated by the server based on the flight path; Displaying a preview video of the described flight route.

73. A system, characterized in that, Includes an electronic device and a remote control as described in any one of claims 37 to 57 or 58 to 72, wherein the remote control is communicatively connected to the electronic device.

74. A computer storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 21 or 22 to 36.

Images