Control method and system, display method and system, device, and storage medium

Abstract

A control method and system, a display method and system, a device, and a storage medium. The control method comprises: during the operation of an aircraft, acquiring feature point information of a physical environment where the aircraft is located (S11); generating a target picture on the basis of the feature point information, wherein the target picture uses feature points to represent said physical environment (S12); acquiring target information during the operation of the aircraft, wherein the target information comprises real-time position information of the aircraft (S13); on the basis of the target information, generating a virtual identifier for display, wherein the display position of the virtual identifier on the target picture can indicate the relative position of the aircraft in said physical environment, and the display position of the virtual identifier in the target picture changes with the real-time position information of the aircraft (S14); and displaying, on a user interface, the target picture and the virtual identifier superimposed on the target picture, wherein the target picture can display, in real time, height changes of the virtual identifier relative to feature points of the ground, and / or the number of feature points displayed in the target picture changes with real-time height information of the aircraft (S15).

Description

Control and display methods and systems, devices and storage media Technical Field

[0001] This application relates to the field of aircraft technology, and in particular to a control and display method and system, device and storage medium. Background Technology

[0002] When operating an aircraft, operators need to refer to the surrounding environment. In related technologies, the aircraft transmits image data of its environment back to its control terminal, allowing the operator to control the aircraft based on the displayed images. However, image data often fails to clearly reflect the relative position of the aircraft to objects in its environment, resulting in a poor user experience and increasing the risk of safety accidents.

[0003] Summary of the Invention

[0004] In a first aspect, this application provides a control method, comprising: actively emitting a detection beam to acquire feature point information of the physical environment in which the aircraft is located during the operation of the aircraft; generating a target image for display based on the feature point information, wherein the target image uses feature points to characterize the physical environment in which the aircraft is located; acquiring target information during the operation of the aircraft, wherein the target information includes the real-time position information of the aircraft; generating a virtual identifier for display based on the target information of the aircraft, wherein the display position of the virtual identifier on the target image can indicate the relative position of the aircraft in its physical environment, and the display position of the virtual identifier on the target image changes with the change of the real-time position information of the aircraft; and displaying the target image and the virtual identifier superimposed on the target image on a user interface; wherein the target image can display the real-time height change of the virtual identifier relative to the feature points on the ground; and / or the number of feature points displayed in the target image changes with the change of the real-time altitude information of the aircraft.

[0005] Secondly, this application provides a display method, comprising: acquiring a target image and a virtual identifier superimposed on the target image during the operation of an aircraft; displaying the target image and the virtual identifier superimposed on the target image in a user interface in real time; wherein the target image uses feature points to characterize the physical environment in which the aircraft is located, the feature points are acquired based on detection signals actively sent by the aircraft's sensors, the virtual identifier is generated based on the target information of the aircraft, the target information includes real-time position information of the aircraft during operation, the display position of the virtual identifier on the target image can indicate the relative position of the aircraft in its physical environment, the display position of the virtual identifier on the target image changes with the change of the real-time position information of the aircraft, wherein the target image can display the height change of the virtual identifier relative to the feature points on the ground in real time; and / or the number of feature points displayed in the target image changes with the change of the real-time altitude information of the aircraft.

[0006] Thirdly, this application provides a control method, comprising: actively emitting a detection beam to acquire feature point information of the physical environment in which the aircraft is located during the operation of the aircraft; generating a target image for display based on the feature point information, wherein the target image uses feature points to characterize the physical environment in which the aircraft is located; acquiring target information during the operation of the aircraft, wherein the target information includes the real-time position information of the aircraft; generating a virtual identifier for display based on the target information of the aircraft, wherein the display position of the virtual identifier on the target image can indicate the relative position of the aircraft in its physical environment, and the display position of the virtual identifier on the target image changes with the change of the real-time position information of the aircraft; and sending the virtual identifier and the target image to the control terminal of the aircraft, so that the user interface of the control terminal can display the target image and the virtual identifier superimposed on the target image; wherein the target image can display the real-time height change of the virtual identifier relative to the feature points on the ground; and / or the number of feature points displayed in the target image changes with the change of the real-time altitude information of the aircraft.

[0007] Fourthly, this application provides a control method, comprising: acquiring feature point information of the physical environment in which the aircraft is located by actively transmitting detection signals from the aircraft's sensors; generating a target image for display based on the feature point information, wherein the target image uses feature points to characterize the physical environment in which the aircraft is located; acquiring target information during the operation of the aircraft, wherein the target information includes the real-time position information of the aircraft; generating a virtual identifier for display based on the target information of the aircraft, wherein the display position of the virtual identifier on the target image can indicate the relative position of the aircraft in its physical environment, and the display position of the virtual identifier on the target image changes with the change of the real-time position information of the aircraft; and displaying the target image and the virtual identifier superimposed on the target image on a user interface; wherein the target image includes a first region that has been detected and a second region that has not been detected, and the relative layout of the first region and the second region changes with the change of the real-time altitude information of the aircraft.

[0008] Fifthly, this application provides a display method, comprising: acquiring a target image and a virtual identifier superimposed on the target image during the operation of an aircraft; and displaying the target image and the virtual identifier superimposed on the target image in a user interface in real time; wherein the target image uses feature points to characterize the physical environment in which the aircraft is located, the feature points being acquired based on detection signals actively transmitted by the aircraft's sensors, the virtual identifier being generated based on target information of the aircraft, the target information including real-time position information of the aircraft during its operation, the display position of the virtual identifier on the target image indicating the relative position of the aircraft in its physical environment, the display position of the virtual identifier on the target image changing with the change of the aircraft's real-time position information, wherein the target image includes a first region that has been detected and a second region that has not been detected, the relative layout of the first region and the second region changing with the change of the aircraft's real-time altitude information.

[0009] Sixthly, this application provides a control method, comprising: acquiring feature point information of the physical environment in which the aircraft is located by actively transmitting detection signals from the aircraft's sensors; generating a target image for display based on the feature point information, wherein the target image uses feature points to characterize the physical environment in which the aircraft is located; acquiring target information during the operation of the aircraft, wherein the target information includes the real-time position information of the aircraft; generating a virtual identifier for display based on the target information of the aircraft, wherein the display position of the virtual identifier on the target image can indicate the relative position of the aircraft in its physical environment, and the display position of the virtual identifier on the target image changes with the change of the real-time position information of the aircraft; and sending the virtual identifier and the target image to the control terminal of the aircraft, so that the user interface of the control terminal can display the target image and the virtual identifier superimposed on the target image; wherein the target image includes a first region that has been detected and a second region that has not been detected, and the relative layout of the first region and the second region changes with the change of the real-time altitude information of the aircraft.

[0010] In a seventh aspect, this application provides a control method, comprising: actively emitting a detection beam to acquire feature point information of the physical environment in which the aircraft is located during the operation of the aircraft; generating a target image for display based on the feature point information, wherein the target image uses feature points to characterize the physical environment in which the aircraft is located; acquiring target information during the operation of the aircraft, wherein the target information includes the real-time position information of the aircraft; generating a virtual identifier for display based on the target information of the aircraft, wherein the display position of the virtual identifier on the target image can indicate the relative position of the aircraft in its physical environment, and the display position of the virtual identifier on the target image changes with the change of the real-time position information of the aircraft; and displaying the target image and the virtual identifier superimposed on the target image on a user interface; wherein the user's viewing angle of the target image includes an upward viewing angle, and the user can observe the bottom surface of the virtual identifier from the upward viewing angle.

[0011] Eighthly, this application provides a display method, comprising: acquiring a target image and a virtual identifier superimposed on the target image during the operation of an aircraft; and displaying the target image and the virtual identifier superimposed on the target image in a user interface in real time; wherein the target image uses feature points to characterize the physical environment in which the aircraft is located, the feature points are acquired based on detection signals actively transmitted by the aircraft's sensors, the virtual identifier is generated based on target information of the aircraft, the target information includes real-time position information of the aircraft during operation, the display position of the virtual identifier on the target image can indicate the relative position of the aircraft in its physical environment, the display position of the virtual identifier on the target image changes with the change of the real-time position information of the aircraft, and the user's viewing angle of the target image includes an upward viewing angle, under which the user can observe the bottom surface of the virtual identifier.

[0012] Ninthly, this application provides a control method, comprising: actively emitting a detection beam to acquire feature point information of the physical environment in which the aircraft is located during the operation of the aircraft; generating a target image for display based on the feature point information, wherein the target image uses feature points to characterize the physical environment in which the aircraft is located; acquiring target information during the operation of the aircraft, wherein the target information includes the real-time position information of the aircraft; generating a virtual identifier for display based on the target information of the aircraft, wherein the display position of the virtual identifier on the target image can indicate the relative position of the aircraft in its physical environment, and the display position of the virtual identifier on the target image changes with the change of the real-time position information of the aircraft; and sending the virtual identifier and the target image to a control terminal of the aircraft, so that the user interface of the control terminal can display the target image and the virtual identifier superimposed on the target image; wherein the user viewing angle of the target image includes a downward viewing angle, and the user can observe the bottom surface of the virtual identifier from the downward viewing angle.

[0013] In a tenth aspect, this application provides a computer device, including a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and, when executing the computer program, to implement the method described in any one of the first to ninth aspects.

[0014] In one aspect, this application provides a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the steps of the method described in any one of the first to ninth aspects.

[0015] In a twelfth aspect, this application provides a system comprising: at least one processor; and at least one memory including computer program code, wherein the at least one memory and the computer program code are configured together with the at least one processor such that the system is at least capable of performing the methods described in any one of the first to ninth aspects. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and constitute a part of this application, illustrate embodiments consistent with this application and, together with the description, serve to explain the technical solutions of this application.

[0017] Figure 1 is a schematic diagram of an aircraft according to an embodiment of this application.

[0018] Figure 2 is a schematic diagram of a control terminal according to an embodiment of this application.

[0019] Figure 3 is a flowchart of a control method according to an embodiment of this application.

[0020] Figures 4, 5, 6 and 7 are schematic diagrams of the target image when the aircraft is in different positions according to an embodiment of this application.

[0021] Figure 8 is a schematic diagram of a target screen including a virtual trajectory according to an embodiment of this application.

[0022] Figure 9 is a schematic diagram of a user's viewing perspective according to an embodiment of this application.

[0023] Figures 10A and 10B are schematic diagrams of a display window on a user interface according to an embodiment of this application.

[0024] Figures 11A and 11B are schematic diagrams of the display window on the user interface of another embodiment of this application.

[0025] Figure 12 is a schematic diagram of the overall architecture of an embodiment of this application.

[0026] Figure 13 is a schematic diagram of a display method corresponding to Figure 3 according to an embodiment of this application.

[0027] Figure 14 is a flowchart of another control method corresponding to Figure 3 according to an embodiment of this application.

[0028] Figure 15 is a flowchart of a control method according to another embodiment of this application.

[0029] Figure 16 is a schematic diagram of a display method corresponding to Figure 14 according to an embodiment of this application.

[0030] Figure 17 is a flowchart of another control method corresponding to Figure 14 according to an embodiment of this application.

[0031] Figure 18 is a flowchart of a control method according to another embodiment of this application.

[0032] Figure 19 is a schematic diagram of a display method corresponding to Figure 18 according to an embodiment of this application.

[0033] Figure 20 is a flowchart of another control method corresponding to Figure 18 according to an embodiment of this application. Detailed Implementation

[0034] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0035] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items. Additionally, the term “at least one” herein means any combination of at least two of any one or more of a plurality.

[0036] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0037] To enable those skilled in the art to better understand the technical solutions in the embodiments of this application, and to make the above-mentioned objectives, features and advantages of the embodiments of this application more apparent and understandable, the technical solutions in the embodiments of this application will be further described in detail below with reference to the accompanying drawings.

[0038] The aircraft in this application embodiment, according to their configuration, may include rotorcraft, fixed-wing aircraft, or hybrid fixed-wing-rotor aircraft, wherein the rotorcraft may be a single-rotor aircraft or a multi-rotor aircraft. The aircraft in this application embodiment, according to their flight purpose, may include aerial photography drones, agricultural drones, surveying drones, logistics drones, etc. The aircraft in this application embodiment, according to their control method, may include unmanned aircraft and manned aircraft. This application embodiment does not impose any particular limitation on the type of aircraft.

[0039] The embodiments of this application will be described below using a multi-rotor drone as an example, in conjunction with the accompanying drawings. Figure 1 shows a schematic diagram of the structure of a flight system 100 according to an embodiment of this disclosure. The flight system 100 may include an aircraft 110 and a control terminal 140. The aircraft 110 may include a power system 150, a flight control system 160, an energy system 170, a frame, and a gimbal 120 mounted on the frame. The aircraft 110 can wirelessly communicate with the control terminal 140. The aircraft 110 may be various types of drones, such as agricultural drones or industrial application drones, and has the need for cyclical operations.

[0040] The frame may include a fuselage and landing gear (also known as landing gear). The fuselage may include a center frame and one or more arms connected to the center frame, with the arms extending radially from the center frame. The landing gear is connected to the fuselage and serves to provide support during the landing of the aircraft 110.

[0041] The power system 150 may include one or more electronic speed controllers (ESCs) 151, one or more propellers 153, and one or more motors 152 corresponding to the propellers 153. The motors 152 are connected between the ESCs 151 and the propellers 153, and the motors 152 and propellers 153 are mounted on the arms of the aircraft 110. The ESCs 151 receive drive signals generated by the flight control system 160 and provide drive current to the motors 152 according to the drive signals to control the rotational speed of the motors 152. The motors 152 drive the propellers to rotate, thereby providing power for the flight of the aircraft 110, enabling the aircraft 110 to achieve one or more degrees of freedom of motion. In some embodiments, the aircraft 110 may rotate about one or more rotation axes. For example, these rotation axes may include a roll axis, a yaw axis, and a pitch axis. It should be understood that the motors 152 may be DC motors or AC motors. Additionally, the motors 152 may be brushless motors or brushed motors.

[0042] The flight control system 160 may include a flight controller 161 and one or more sensors 162 of the aircraft 110. The sensors 162 of the aircraft 110 may include sensors for measuring the attitude information of the aircraft 110. This attitude information refers to the position and state information of the aircraft 110 in space, such as three-dimensional position, three-dimensional angle, three-dimensional velocity, three-dimensional acceleration, and three-dimensional angular velocity. The sensors used to measure the attitude information of the aircraft 110 may include, but are not limited to, at least one of the following: gyroscope, ultrasonic sensor, electronic compass, inertial measurement unit (IMU), visual sensor, global navigation satellite system, and barometer. The global navigation satellite system may be the Global Positioning System (GPS). The sensors 162 of the aircraft 110 may also include sensors for sensing environmental information, such as lidar, visual sensor, infrared sensor, and ultrasonic sensor. The flight controller 161 is used to control the flight state of the aircraft 110; for example, it can control the flight of the aircraft 110 based on the attitude information measured by the sensors 162 and the sensed environmental information. It should be understood that the flight controller 161 can control the aircraft 110 according to pre-programmed instructions, or it can control the aircraft 110 in response to one or more remote control signals from the control terminal 140.

[0043] The gimbal 120 may include a motor 122. The gimbal can be used to carry an imaging device 123. The flight controller 161 can control the movement of the gimbal 120 via the motor 122. Optionally, as another embodiment, the gimbal 120 may also include a controller for controlling the movement of the gimbal 120 by controlling the motor 122. It should be understood that the gimbal 120 may be independent of the aircraft 110 or may be part of the aircraft 110. It should be understood that the motor 122 may be a DC motor or an AC motor. Additionally, the motor 122 may be a brushless motor or a brushed motor.

[0044] The imaging device 123 may be, for example, a camera or video camera, a device used to capture images. The imaging device 123 can communicate with the flight controller 161 and take pictures under the control of the flight controller 161. In this embodiment, the imaging device 123 includes at least a photosensitive element, such as a complementary metal-oxide-semiconductor (CMOS) sensor or a charge-coupled device (CCD) sensor. It is understood that the imaging device 123 can also be directly fixed to the aircraft 110, thus the gimbal 120 can be omitted.

[0045] The energy system 170 may include one or more batteries and a battery management system (BMS), wherein the batteries can be used to power the power system 150, the flight control system 160, the gimbal 120 and loads on the gimbal 120 (e.g., imaging device 123), and the battery management system is used to manage and control the charging and discharging process of the batteries.

[0046] It should be understood that the naming of the components of the flight system 100 described above is for identification purposes only and should not be construed as a limitation of this disclosure. In addition to the components mentioned above, the aircraft 110 may also include other components, such as a graphics processor, an encoder, a media center, etc. (not shown). The graphics processor is used to perform image processing on images acquired by the imaging device 123 or images obtained by other means; the encoder is used to encode the images output by the graphics processor into a bitstream and send it to other devices; and the media center is used to provide certain data processing capabilities.

[0047] Figure 2 shows a schematic diagram of the control terminal 140. The control terminal 140 can be a smart terminal such as a mobile phone or tablet computer, or it can be a remote controller dedicated to the aircraft 110. The control terminal 140 may include a communication device 141, a display device 142, and an input device 143. The communication device 141 can communicate with the aircraft 110 based on wireless communication methods (such as WiFi, Bluetooth, 4G, 5G, etc.). The display device 142 can display the image transmission data sent by the aircraft 110, the status information of the aircraft 110, the load information on the aircraft 110, flight restriction information, sensor data, and / or alarm information, etc. The number of input devices 143 can be greater than or equal to one, and each input device 143 can be located on the top, side, or other positions of the control terminal 140. The types of input devices 143 include, but are not limited to, buttons, sliders, dials, knobs, virtual control components on a touch screen, gesture input devices, and / or voice input devices, etc. Users can input control commands through the input device, which can be sent to the aircraft 110 through the communication device 141 to control the status information of the aircraft 110 or the status information of the load on the aircraft.

[0048] In related technologies, aircraft transmit image data of their surroundings back to their control terminal, allowing operators to control the aircraft based on the displayed images. However, image data often fails to intuitively reflect the relative positions of the aircraft and objects in its environment, resulting in a poor user experience. This is especially problematic in complex flight scenarios where safety hazards may exist in multiple directions, making it difficult for operators to monitor all areas simultaneously and increasing the risk of accidents.

[0049] Based on this, this application provides a control method, as shown in Figure 3, the method comprising:

[0050] Step S11: During the operation of the aircraft 110, actively transmit a detection beam to obtain feature point information of the physical environment in which the aircraft 110 is located;

[0051] Step S12: Based on feature point information, generate a target image for display, wherein the target image uses feature points to characterize the physical environment in which the aircraft 110 is located;

[0052] Step S13: Obtain target information during the operation of the aircraft 110, wherein the target information includes the real-time position information of the aircraft 110;

[0053] Step S14: Based on the target information of the aircraft 110, generate a virtual identifier for display. The display position of the virtual identifier on the target screen indicates the relative position of the aircraft 110 in its physical environment. The display position of the virtual identifier on the target screen changes with the real-time position information of the aircraft 110; and

[0054] Step S15: Display the target screen and the virtual logo superimposed on the target screen on the user interface;

[0055] The target screen can display the real-time changes in the height of the virtual marker relative to the ground feature points; and / or the number of feature points displayed in the target screen changes with the real-time altitude information of the aircraft.

[0056] The above method generates a third-person target view in real time during the operation of the aircraft. This view is used to characterize the physical environment in which the aircraft is located through feature points. Virtual markers are generated in real time to indicate the relative position of the aircraft in its physical environment. The target view and the virtual markers superimposed on the target view are displayed on the user interface. This allows users to intuitively perceive the relative position of the aircraft in the physical environment and potential safety hazards in multiple directions. Furthermore, the target view can display the height changes of the virtual markers relative to the feature points on the ground in real time. This helps users to more intuitively observe the relative height relationship between the aircraft and objects in the surrounding physical environment, thus better assisting in safe flight. And / or the number of feature points displayed in the target view changes with the flight altitude of the aircraft, which helps users to intuitively observe the changes in the surrounding physical environment caused by the change in the aircraft's flight altitude, thus better assisting in safe flight and improving the user's operating experience.

[0057] In step S11, feature point information of the physical environment in which the aircraft 110 is located can be obtained through the sensor 162 on the aircraft 110 used for sensing environmental information. The feature point information includes feature point information collected by the sensor 162 of the aircraft 110 at the current moment. In some embodiments, the feature point information includes feature point information collected by the sensor 162 of the aircraft 110 at the current moment and feature point information collected by the sensor 162 of the aircraft 110 at previous moments. The feature point information collected at previous moments is used to supplement the detection blind spots of the sensor at the current moment, for example, by buffering historical feature point information.

[0058] In some embodiments, sensor 162 may include a lidar. Further, sensor 162 employs an omnidirectional lidar. An omnidirectional lidar is a lidar capable of scanning the environment in all 360 degrees; it can be implemented using a rotating lidar capable of 360-degree scanning, or it can be a combination of multiple lidar units that detect in different directions. The omnidirectional lidar can perform a real-time omnidirectional scan of the environment in which the aircraft 110 is located, obtaining feature point information of the physical environment in which the aircraft 110 is located. In this case, the feature point information collected in real time by the omnidirectional lidar at the current moment can be determined as the feature point information in step S11.

[0059] In some embodiments, if the sensor 162 is unable to perform a 360-degree real-time omnidirectional scan due to limitations in its detection range, or if the detection signal is obstructed or interfered with, it needs to be supplemented based on feature point information collected at previous moments. For example, the area behind the virtual identifier in the target image (the area opposite to the direction of movement) can be supplemented using feature point information collected at previous moments.

[0060] The sensor 162 in this embodiment may also include other types of sensors such as millimeter-wave radar. As long as it can actively emit a detection beam to obtain feature point information of the physical environment in which the aircraft 110 is located, this application does not impose any restrictions on it.

[0061] Feature point information may include, but is not limited to, the spatial coordinates, normal vectors, and semantic information corresponding to the feature points. The spatial coordinates define the location of the feature point, the normal vector describes the surface orientation of the feature point, and the semantic information describes the semantic label of the object to which the feature point belongs. In some embodiments, the semantic label characterizes whether the feature point is on the ground. Feature points on the ground include, but are not limited to, feature points corresponding to objects such as vegetation, sand, landmarks, and manhole covers. Feature points not on the ground include, but are not limited to, feature points corresponding to objects such as buildings, vehicles, and trees. In other embodiments, the semantic label characterizes whether the object to which the feature point belongs is an obstacle. Based on the feature point information, semantic labels for different semantic objects can be obtained, and based on the semantic labels, obstacle feature points and non-obstacle feature points can be identified. An obstacle refers to an object that may affect the safety and movement of the aircraft 110. Semantic information can be obtained by semantic recognition algorithms through semantic recognition of feature points. By obtaining semantic information, noise can be removed, the background can be distinguished from various different obstacles, and different rendering, downsampling, and storage strategies can be formulated according to the application scenario. For example, in scenarios involving spraying drones, this application can uniformly treat objects such as utility poles and power lines as obstacles, using 6x6x6cm... 3 A refined model is obtained by voxel downsampling, where only one obstacle feature point is retained within a 6cm×6cm×6cm space voxel. The coordinates of this obstacle feature point can be adjusted to the center of the voxel to optimize the display effect. For background fields and fruit tree canopies, a 30x30cm voxel is used. 3 The voxel downsampling method retains only one non-obstacle feature point within a 30cm×30cm×30cm space. The coordinates of this obstacle feature point can also be adjusted to the voxel center to optimize the display effect; this achieves integration of as much historical feature point information as possible. Distance shading is used for obstacles to prompt the operator to pay attention to obstacles that are closer; relative height shading is used for the background to improve the distinguishability of background information.

[0062] In step S12, feature point information can be used to characterize the physical environment in which the aircraft 110 is located, obtaining a target image (such as a three-dimensional target image). This target image can be a third-person perspective image, that is, it can be assumed to be the image "seen" by an "observer" outside the aircraft 110 when observing the aircraft and its surrounding environment. In this way, the user can more intuitively perceive the relative position of the aircraft in its physical environment, so as to operate the aircraft more safely. The target image can be presented to the user in a stereoscopic view to assist in the operation of the aircraft 110. This stereoscopic view can serve as a supplement to the traditional first-person perspective (FPV) real-time preview.

[0063] In some embodiments, the target image can directly use feature point information to represent objects in the physical environment, or the feature point information can be processed to use the processed specific point information to represent objects in the physical environment, for example, by using feature point clustering or downsampling.

[0064] For example, if the number of feature points collected by the sensor 162 on the aircraft 110 is relatively large, the feature points collected by the sensor 162 can be downsampled to filter out some of them, thereby reducing the bandwidth for data processing and storage. The feature points included in the target image can be the downsampled feature points.

[0065] In some embodiments, different feature points in the target image can be rendered using different rendering strategies. For example, a first rendering strategy can be used to render obstacle feature points in the target image, and a second rendering strategy, different from the first, can be used to render non-obstacle feature points in the target image. For instance, obstacles can be specified through system presets; for example, the system defaults to designating objects of a specific semantic type as obstacles, such as considering a wire as an obstacle. In some cases, obstacle semantic types can also be added and deleted by the user. Users can add or delete certain obstacle semantic types based on the system presets, according to actual operational needs, to better adapt to the user's actual operational scenarios and requirements.

[0066] In some embodiments, the first rendering strategy and the second rendering strategy can be used to indicate the rendering color of feature points.

[0067] Optionally, the first rendering strategy includes a distance rendering strategy, which instructs feature points at different distances from the virtual identifier to use different rendering colors. For example, obstacle feature points less than 10 meters from the aircraft 110 are rendered in purple, obstacle feature points greater than or equal to 10 meters but less than 20 meters from the aircraft 110 are rendered in orange, and obstacle feature points greater than or equal to 20 meters from the aircraft 110 are rendered in yellow. By using a distance rendering strategy for obstacle feature points, users can easily distinguish the distance of obstacles from the aircraft 110, serving both a color warning function and facilitating more accurate obstacle avoidance maneuvers for the aircraft 110.

[0068] Optionally, the second rendering strategy includes a height rendering strategy, which instructs feature points at different heights from the virtual identifier to use different rendering colors. For example, non-obstacle feature points with a height difference of less than 5 meters from the aircraft 110 are rendered in dark blue, non-obstacle feature points with a height difference of 5 meters or more but less than 10 meters from the aircraft 110 are rendered in light blue, and non-obstacle feature points with a height difference of 10 meters or more from the aircraft 110 are rendered in green. By using a height rendering strategy for non-obstacle feature points, users can easily distinguish non-obstacle feature points at different heights, which helps users intuitively perceive the three-dimensional layers of non-obstacles and makes it easier to distinguish the outline of non-obstacles.

[0069] It is understood that the above embodiments are merely illustrative. In practical applications, the segmented ranges of the rendering color and the corresponding rendering color for each range can be set according to actual needs.

[0070] In some embodiments, the first rendering strategy and the second rendering strategy can be used to indicate the rendering density of feature points. Optionally, the rendering density of obstacle feature points is greater than the rendering density of non-obstacle feature points. For example, the first rendering strategy includes rendering obstacle feature points with a first rendering density, and the second rendering strategy includes rendering non-obstacle feature points with a second rendering density, where the first rendering density is greater than the second rendering density. During the flight of the aircraft 110, the impact of obstacles on the aircraft 110 is generally greater than the impact of non-obstacles. Therefore, using a higher rendering density to render obstacle feature points provides more obstacle information and provides a better visual warning effect; while using a lower rendering density for non-obstacles can effectively reduce resource consumption and time consumption during the rendering process, saving system resources. Intuitively, this means that the rendering of obstacle feature points in the target image is relatively dense, while the rendering of non-obstacle feature points is relatively sparse.

[0071] In some embodiments, the first rendering strategy and the second rendering strategy can be used to indicate the upper limit of rendering obstacle feature points and non-obstacle feature points. For example, the first rendering strategy is used to indicate the upper limit of the rendering ratio of obstacle feature points, or the upper limit of the number of obstacle feature points rendered; the second rendering strategy is used to indicate the upper limit of the rendering ratio of non-obstacle feature points, or the upper limit of the number of non-obstacle feature points rendered. Optionally, the upper limit of the rendering ratio of obstacle feature points is higher than the upper limit of the rendering ratio of non-obstacle feature points, for example, 7:3. In this way, more processing resources used in the rendering process can be allocated to obstacle feature points, allowing users to observe obstacle feature points in the target image more comprehensively and accurately, and prioritizing the rendering effect of obstacle feature points. At the same time, by setting the upper limit of the number or rendering ratio of obstacle feature points and non-obstacle feature points, the packet loss problem in unreliable transmission links can be overcome, and the defect of feature point information loss can be overcome. This ensures the robustness of the auxiliary view (i.e., the target image), and users can directly rely on the auxiliary view to work, because when the transmission link bandwidth is limited, the worst case is only a reduction in the video stream bitrate.

[0072] In step S13, target information during the operation of aircraft 110 can be acquired. This target information may include the real-time position information of aircraft 110. The real-time position information of aircraft 110 can be obtained by using sensors on aircraft 110 to detect its position. Alternatively, the real-time position information of aircraft 110 can be acquired by other devices besides aircraft 110 (such as other aircraft around aircraft 110, or airport equipment or cloud platforms communicating with aircraft 110). After acquiring the real-time position information of aircraft 110, these other devices can transmit the real-time position information to aircraft 110 through a communication connection, or broadcast the real-time position information of aircraft 110 so that aircraft 110 can obtain its own real-time position information. A combination of the above two methods can also be used to acquire the real-time position information of aircraft 110. For example, sensors used to detect the real-time location information of the aircraft 110 may include GPS sensors, RTK (Real-Time Kinematic) or visual sensors, etc.

[0073] The aircraft 110 is capable of attitude changes around at least one of its pitch, roll, and yaw angles. Correspondingly, the virtual identifier is also capable of attitude changes around at least one of its pitch, roll, and yaw angles. In addition to the real-time position information of the aircraft 110, the target information may also include the real-time attitude information of the aircraft 110. The real-time attitude information may include at least one of the aircraft 110's pitch, roll, and yaw angles. Specifically, when the real-time attitude information is the aircraft 110's yaw angle, this real-time attitude information may also be referred to as the aircraft 110's real-time orientation information. The real-time attitude information can be detected in real-time by the aircraft 110's sensor 162. By acquiring the real-time attitude information, a virtual identifier corresponding to the aircraft 110's actual attitude can be generated. For example, the sensor used to detect the real-time attitude information of the aircraft 110 may include an IMU sensor.

[0074] In step S14, a virtual identifier can be generated based on the target information of the aircraft 110, and the virtual identifier can be displayed on the target screen. Figure 4 shows a schematic diagram of the target screen in some embodiments. As shown in Figure 4, the target screen includes a virtual identifier 401, ground feature points 402, and non-ground feature points 403. For example, the ground feature points are the feature points corresponding to grass in the physical space in the target screen, and the non-ground feature points are the feature points corresponding to power transmission towers in the physical space in the target screen.

[0075] The virtual identifier can be used to represent the aircraft 110. For example, the virtual identifier can include a simulation model of the aircraft 110. This simulation model can be obtained by scaling the physical entity of the aircraft 110 according to a preset ratio. The size ratio between the simulation model and the physical entity of the aircraft 110 is the same as the size ratio between the displayed object in the target screen and its corresponding entity object in the physical environment. This ensures that the ratio between the rendered virtual identifier and the displayed object in the target screen remains consistent with the proportions of the physical world. For example, assuming that the entity object in the physical environment includes a utility pole, and the target screen includes a display object corresponding to the utility pole, if the size ratio between the display object corresponding to the utility pole in the target screen and the utility pole in the physical environment is 1:10, then the size ratio between the simulation model and the physical entity of the aircraft 110 is also 1:10.

[0076] When rendering the simulation model of aircraft 110, to enhance the realism of the image, the rendering parameters of the simulation model can correspond to the characteristics of the physical entity of aircraft 110. These characteristics can include, but are not limited to, shape features, including but not limited to, outline shape, size, color, and / or material. Correspondingly, the rendering parameters of the simulation model can also include outline shape, size, color, and / or material. The outline shape reflects the external structure of aircraft 110; determining the rendering parameters of the simulation model based on the outline shape features of the physical entity of aircraft 110 facilitates quick identification of the aircraft by the user. Size features reflect the size of aircraft 110; determining the rendering parameters of the simulation model based on the size features of the physical entity of aircraft 110 makes the virtual identifier appear more harmonious in the target image, ensuring that the relative size relationship between the virtual identifier and other elements in the target image conforms to the relative size relationship between the aircraft and environmental objects in the physical environment. Determining the rendering parameters of the simulation model based on the color and material of the physical entity of aircraft 110 also effectively improves the realism of the rendering result. For example, if the aircraft is a quadcopter, the virtual identifier will also be a quadcopter model; as another example, if the aircraft is a coaxial rotorcraft, the virtual identifier will also be a coaxial rotorcraft model. The simulation mode can be three-dimensional.

[0077] Furthermore, the virtual identifier can also include a simulation model of the payload of the aircraft 110. For example, when the aircraft 110 carries payloads such as cameras, first-aid kits, and de-icing sticks, the rendered virtual identifier includes the portion corresponding to the aircraft 110 and the portion corresponding to the payload. Through the above rendering method, the rendered virtual identifier reflects the payload carried by the aircraft 110, thereby further improving the realism of the virtual identifier.

[0078] It is understood that the above is merely an illustrative example and is not intended to limit this application. In other examples, fixed icons (such as arrow icons) can also be used as virtual identifiers to reduce the complexity of rendering virtual identifiers.

[0079] The display position of the virtual marker on the target screen can indicate the relative position of the aircraft 110 in its physical environment. Referring to Figure 4, the target screen includes feature points corresponding to grass and feature points corresponding to power transmission towers. If the virtual marker is located above the feature point corresponding to the grass and directly facing the feature point corresponding to the power transmission tower in the target screen, it indicates that the aircraft 110 is on a grassy area in its physical environment and is directly facing the power transmission tower.

[0080] The virtual marker's position on the target screen changes according to the real-time position information of the aircraft 110. For example, referring to Figures 4 and 5, when the aircraft 110 moves from a position directly facing the power transmission tower to the right of the tower in its environment, the virtual marker's position on the target screen will also move accordingly from the position of the feature point 403 directly facing the tower to the right of the feature point 403. In this way, the user can intuitively distinguish the real-time position changes of the aircraft 110 on the target screen, facilitating the user's operation and control of the aircraft 110.

[0081] In step S15, the target image and a virtual icon superimposed on the target image can be displayed on the user interface. The user interface can be the user interface on the control terminal of the aircraft 110, the user interface of an airport or cloud platform communicating with the aircraft 110, or the user interface on the aircraft 110 itself. The target image and the virtual icon can be rendered separately first, then the virtual icon can be superimposed on the target image, and finally the target image and the virtual icon superimposed on the target image can be displayed on the user interface.

[0082] In some embodiments, the target screen can display the real-time height changes of the virtual marker relative to feature points on the ground. For example, the target screen can display the real-time height changes of the bottom surface of the virtual marker relative to feature points on the ground, or it can display the height changes of specific components of the virtual marker (such as the rotor or load of aircraft 110) relative to feature points on the ground. For example, during the ascent of aircraft 110, the height of the virtual marker relative to feature points on the ground will increase accordingly, and the target screen can display this increase in height in real time. Assuming that the initial height of aircraft 110 in physical space is close to the ground, the target screen with the virtual marker superimposed is shown in Figure 5. If aircraft 110 ascends in physical space, its real-time height in physical space will be further away from the ground, and the target screen with the virtual marker superimposed is shown in Figure 6. By comparing Figures 5 and 6, it can be seen that when the height of aircraft 110 increases, the height of the virtual marker relative to feature points on the ground in the target screen also increases. Similarly, during the descent of aircraft 110, the height of the virtual marker relative to feature points on the ground will decrease accordingly, and the target screen can display this decrease in height in real time.

[0083] In some embodiments, a prompt message may be displayed on the target screen to indicate the real-time altitude information (such as the absolute value of the altitude) and / or altitude changes of the aircraft 110 (such as the relative altitude value of the aircraft 110 rising or falling relative to a certain reference altitude).

[0084] In some embodiments, the number of feature points displayed in the target image can vary with the real-time altitude information of the aircraft 110. Referring to Figures 5 and 7, as the altitude of the aircraft 110 increases, the sensing range of the sensor 162 increases, thereby collecting more feature points, and thus the number of feature points displayed in the target image also increases. Conversely, as the altitude of the aircraft 110 decreases, the sensing range of the sensor 162 decreases, thereby collecting fewer feature points, and thus the number of feature points displayed in the target image also decreases.

[0085] In addition to overlaying virtual identifiers on the target screen, this application can also overlay other auxiliary information on the target screen to output a reliable auxiliary view to the user, thereby improving the efficiency and safety of manipulating work units in complex work scenarios.

[0086] In some embodiments, referring to FIG8, in addition to overlaying virtual identifiers on the target screen, a virtual trajectory 404 of the aircraft 110 can also be overlaid and displayed on the target screen. The virtual trajectory 404 can indicate the relative position of the aircraft 110's motion trajectory in the physical environment in which the aircraft 110 is located. By overlaying the virtual trajectory 404 on the target screen, the user can intuitively observe the motion trajectory of the aircraft 110 and grasp the direction and speed of the aircraft 110's motion, thereby facilitating the control of the aircraft 110.

[0087] The virtual trajectory 404 superimposed on the target screen can change along with the movement trajectory of the aircraft 110. For example, when the aircraft's movement trajectory is from the first direction to the second direction, as the aircraft 110 gradually moves towards the second direction, the virtual trajectory 404 on the target screen also gradually extends towards the second direction. Alternatively, as the aircraft 110 gradually moves towards the second direction, the trajectory points of the virtual trajectory 404 in the first direction can gradually be eliminated from the target screen. In this way, the effect of the virtual trajectory 404 dynamically changing from the first direction to the second direction can be presented on the target screen, allowing users to intuitively observe the changes in the movement trajectory of the aircraft 110.

[0088] The trajectory of aircraft 110 may include at least one of a historical trajectory and a planned trajectory. The historical trajectory includes the trajectory of aircraft 110 moving backwards for a preset time period from the current moment. Trajectory points of aircraft 110 within the preset time period can be cached, and the historical trajectory can be determined based on the cached trajectory points. The planned trajectory includes the planned trajectory of aircraft 110 moving backwards for a preset time period from the current moment. The planned trajectory can be automatically planned by the flight control system 160 based on the status information of aircraft 110 and the environmental information of the environment in which aircraft 110 is located; alternatively, it can be manually planned by the user. The preset time period can be 1 to 3 seconds, and can be set according to actual needs. The user controls the aircraft to move along the planned operating path and takes corresponding countermeasures in advance based on near-field information.

[0089] In some embodiments, the virtual trajectory of the aircraft 110 can be overlaid on the target screen in a preset style. The preset style includes, but is not limited to, color, brightness, width, and / or animation effects. For example, when the preset style includes color, a virtual trajectory of a preset color (such as gray or black) or a virtual trajectory of a color significantly different from the background color in the target screen can be displayed on the target screen. When the preset style includes brightness, a virtual trajectory with a preset brightness or a brightness significantly different from the background brightness in the target screen can be displayed on the target screen. When the preset style includes width, a virtual line of a certain width can be obtained by extending radially from the virtual identifier. When the preset style includes animation effects, a virtual trajectory flashing at a preset frequency can be displayed on the target screen. Besides the cases listed above, the preset style can also be other types of styles, which will not be listed here.

[0090] In some embodiments, the virtual trajectory 404 includes trajectory elements, which include trajectory points and / or trajectory segments. Trajectory points represent the position of the aircraft 110 at a given moment; trajectory points represent the position of the aircraft 110 over a given period of time; and trajectory segments can be line segments formed by connecting multiple trajectory points.

[0091] In some embodiments, the trajectory elements include specific trajectory elements, and the display style of the specific trajectory elements is different from the display style of other trajectory elements besides the specific trajectory elements, so as to make the user notice the specific trajectory elements.

[0092] Specific trajectory elements are used to indicate at least one of the following: motion risk points, motion start points, motion end points, motion turning points, motion interruption points, motion continuation points, operational task points, and non-operational task points. Motion risk points are trajectory points where safety hazards or accident risks may occur during motion. For example, if there are obstacles in the environment where aircraft 110 is located, motion risk points include trajectory points where collisions with obstacles are possible. Motion start points are trajectory points where motion begins, marking the initial stage of the motion process. Motion end points are trajectory points where motion ends, marking the end of the motion process. Motion turning points are trajectory points where a turn or change of course occurs during motion. Motion interruption points are trajectory points where motion is paused or terminated due to malfunctions, weather conditions, emergencies, etc. Motion continuation points are trajectory points where resources are replaced or replenished during motion (such as replacing batteries or refilling pesticides for agricultural drones). Operational task points are trajectory points where operational tasks are performed (such as de-icing, spraying pesticides, or transporting goods). Non-operational task points are trajectory points that do not involve specific operational tasks.

[0093] In some embodiments, specific display elements can be displayed on the target screen using a preset style. Preset styles include, but are not limited to, color, brightness, and / or animation effects. For example, when the preset style includes color, specific display elements can be displayed on the target screen using a preset color (such as yellow, red, or blue), or different colors can be used to display specific and non-specific display elements on the target screen. When the preset style includes brightness, specific display elements can be displayed on the target screen using a preset brightness, or different brightness levels can be used to display specific and non-specific display elements on the target screen. When the preset style includes animation effects, specific display elements that flash at a preset frequency can be displayed on the target screen. Besides the cases listed above, preset styles can also be other types of styles, which will not be listed here.

[0094] Optionally, specific display elements can be used to indicate the operating area of ​​the aircraft 110. For example, when the aircraft 110's operation type is de-icing power transmission lines, the operating area is the area where the power transmission lines that need to be de-iced are located. As another example, when the aircraft 110's operation type is spraying pesticides in farmland, the operating area is the area where the farmland that needs to be sprayed is located.

[0095] Optionally, specific display elements can be used to indicate specific trajectory points of the aircraft 110. These specific trajectory points include any of the following: a start point of motion, a finish point of motion, a turning point of motion, a point of interruption of motion, a point of continuation of motion, a task point, or a non-task point. Specific embodiments of the aforementioned specific trajectory points are detailed above and will not be repeated here.

[0096] Optionally, specific display elements may be used to indicate content related to the collision risk of the aircraft 110.

[0097] The content related to the collision risk of aircraft 110 may include a spatial region extending a predetermined distance from aircraft 110 along its forward direction. This predetermined distance can be between 10 meters and 30 meters. The predetermined distance can be a fixed value or a dynamically changing value. In the example of a dynamically changing predetermined distance, it can be positively correlated with the size of the physical entity of aircraft 110 or with the speed of aircraft 110. This application, referring to the volume and direction of movement of aircraft 110, calculates the spatial region that the work unit will traverse in the next period of time, and identifies the danger zone by highlighting feature points within this region using a rendering strategy, thereby warning the user of the possibility of a collision in the current direction of movement. As the physical size of the aircraft 110 increases, or as its speed increases, the risk of collision also increases. By setting the preset distance to be positively correlated with the size of the aircraft 110 or its speed, sufficient reaction time can be provided for the operator to control the aircraft 110 to avoid obstacles, thereby reducing the risk of collision.

[0098] Alternatively, content related to the collision risk of aircraft 110 may include potential collision obstacles, including static potential collision obstacles (such as buildings, trees, utility poles, etc.) and dynamic potential collision obstacles (such as people or other aircraft around aircraft 110). In cases where the target image includes obstacle feature points and non-obstacle feature points, potential collision obstacles are a special case of the aforementioned obstacles. Potential collision obstacles are obstacles with specific collision risks. For example, potential collision obstacles include those within a spatial region extending a predetermined distance from aircraft 110 along its forward direction, where obstacles within this spatial region have a high collision risk level. For instance, feature points whose preset semantic information is "electric wire" are rendered purple, and obstacle feature points not located within the spatial region extending a predetermined distance from aircraft 110 along its forward direction are rendered purple; obstacle feature points whose preset semantic information is "electric wire" and are further located within the spatial region extending a predetermined distance from aircraft 110 along its forward direction are rendered red.

[0099] Alternatively, content related to collision risk may include the potential direction of collision. For example, when there is an obstacle to the left of aircraft 110, the potential direction of collision may include the left side of the aircraft. For example, a left-pointing arrow may be displayed in the center of the target image.

[0100] Alternatively, content related to collision risk includes potential collision locations. Potential collision locations may include the potential collision site on the side of aircraft 110, and / or the potential collision site on the side of the potential collision obstacle. For example, when there is an obstacle on the left side of aircraft 110, and the left rotor of the aircraft may collide with the top of the obstacle, the potential collision location may include the left rotor of the aircraft, and / or the right side of the obstacle.

[0101] Optionally, specific display features can be used to indicate the area where designated infrastructure (such as airports, communication base stations, etc.) is located. Alternatively, specific display features can be used to indicate areas pre-set or marked by the user. Depending on actual needs, specific display features can also be used to indicate other types of points or areas, which will not be listed here.

[0102] The display style of a specific display element in the target image can differ from the display style of other feature points in the target image. This makes it easier for users to distinguish the specific display element from other feature points in the target image. For example, the specific display element can be displayed in red and other feature points in blue; or, the specific display element can be displayed with higher brightness and other feature points with lower brightness.

[0103] In some embodiments, specific display elements are highlighted in the target screen. For example, the specific display element can be displayed in a more eye-catching color, or a border effect can be added around the specific display element. By highlighting the specific display element, it is easier to attract the user's attention and make the user focus on the specific display element.

[0104] In some embodiments, other feature points in the target image besides specific display elements can also be hidden. For example, specific display elements in the target image can be automatically identified, and other feature points can be automatically hidden based on the identification results. Alternatively, the user can decide whether to hide other feature points to more focusedly display specific display elements that require special attention from the user. For example, a physical control can be provided on the control terminal or a virtual space can be provided on the user interface, allowing the user to operate the aforementioned control to control whether other feature points besides the display elements are hidden or displayed.

[0105] In some embodiments, the user's viewing angle of the target image can be adjusted to allow the user to observe the target image from different perspectives. Considering that the purpose of this application is to assist in control, an orbital control strategy with the center of the aircraft 110 as the center of the sphere can be used to determine the user's viewing angle. Referring to Figure 9, the user's viewing angle includes a level-on viewing angle, a top-down viewing angle, and a bottom-up viewing angle, with a range of approximately 180 degrees or close to 180 degrees. In addition, the viewing angle may also include other viewing angles such as a close-up viewing angle, a long-distance viewing angle, and a side-view viewing angle. The user can freely adjust the viewing angle as needed, such as adjusting it to a level-on viewing angle, a top-down viewing angle, or a side-view viewing angle. For example, the user's viewing angle is the perspective observed from behind the virtual icon.

[0106] In some embodiments, the user's viewing angle can be adjusted around a preset axis of the virtual icon. The preset axis includes at least one of the virtual icon's yaw axis, pitch axis, and roll axis. When the preset axis includes the virtual icon's pitch axis, the user's viewing angle can include at least one of a top-down viewing angle, a level viewing angle, and a bottom-up viewing angle. In some embodiments, in a top-down viewing angle, the user can observe the top surface of the virtual icon; in a bottom-up viewing angle, the user can observe the bottom surface of the virtual icon. In some embodiments, in a top-down viewing angle, the user can observe at least the top surface of the virtual icon's rotor; in a bottom-up viewing angle, the user can observe at least the bottom surface of the virtual icon's rotor.

[0107] In some application scenarios, the upward viewing angle is very advantageous. For example, when navigating narrow passages (such as going through caves, tunnels, or under bridges), or when navigating in situations with overhead obstacles (such as flying under a utility pole or under an indoor ceiling), switching to the upward viewing angle allows users to intuitively perceive the spatial relationship between the aircraft and the objects above, enabling them to better control the aircraft and avoid collisions with overhead obstacles.

[0108] In some embodiments, the user's viewing angle of the target image can be locked relative to a preset axis of the virtual icon, which includes at least one of the yaw axis, pitch axis, and roll axis of the virtual icon. For example, in a spraying drone scenario, this application can lock the yaw angle so that it faces the nose direction and lock the roll angle to 0, allowing the user to freely adjust the viewing angle in the pitch direction.

[0109] In some embodiments, a default viewing angle can be preset. For example, the head-up viewing angle shown in Figure 9 can be set as the default viewing angle. Of course, in other examples, other user viewing angles can also be set as the default viewing angle. Optionally, the default viewing angle can also be changed in response to a user's instruction to change the default viewing angle.

[0110] The system can respond to user adjustment commands and adjust the user's viewing angle. Manually adjusting the user's viewing angle gives the user more autonomy during the adjustment process and makes the adjustment result more in line with the user's personal preferences and habits. In some embodiments, the adjustment command may originate from the user's input operation on the control terminal 140 of the aircraft 110. For example, the control terminal 140 may include preset physical controls (such as buttons, dials, knobs, etc.). The adjustment command may originate from the user's input operation on the aforementioned preset physical controls. The preset physical controls can be individually set controls or controls that reuse other functions. Taking a dial as an example, the user can adjust the viewing angle by rotating the dial. Adjusting the viewing angle using a dial is simple to operate and allows for continuous adjustment to switch to the desired viewing angle. When the preset physical control is a button, the user can also adjust the viewing angle by using different buttons and different operation methods on the buttons (such as single click, double click, long press, or short press). When the preset physical control is a knob, the user can also adjust the viewing angle by rotating the knob. Furthermore, the preset physical controls can also be other types of controls, which will not be listed here.

[0111] In some embodiments, adjustment instructions may originate from user input on the user interface. The user interface may include preset virtual controls, and adjustment instructions may originate from user editing of virtual icons. Preset virtual controls include, but are not limited to, at least one of the following: sliders, input boxes, dials, or icons.

[0112] In some embodiments, the user's viewing angle can also be adjusted in response to an automatically triggered command. Adjusting the user's viewing angle via an automatically triggered command allows for rapid responses in rapidly changing environments, ensuring the user has the best view at critical moments, improving safety, and reducing the user's operational burden.

[0113] The methods for adjusting the user's perspective include, but are not limited to, any of the following:

[0114] (1) The user's viewing angle is adjusted so that the adjusted viewing angle is more conducive to observing factors affecting the maneuvering behavior of the aircraft 110 compared to the original viewing angle. For example, when the aircraft 110 needs to land, an automatic trigger command can be generated to switch the user's viewing angle to a top-down viewing angle, and the user's viewing angle is adjusted to a top-down viewing angle in response to the automatic trigger command. This makes it easier for the user to observe obstacles below the aircraft 110, thereby controlling the aircraft 110 to land safely. As another example, when the aircraft 110 needs to move to the right, a head-up viewing angle facing to the right can be generated, and the user's viewing angle is adjusted to a head-up viewing angle facing to the right in response to the automatic trigger command, so that the user can observe obstacles on the right side of the aircraft 110.

[0115] (2) Adjust the user's viewing angle so that the adjusted viewing angle is more suitable for the proportion of the content displayed in the target screen within the display window compared to the original viewing angle. For example, if the proportion of the content displayed in the target screen within the display window is less than a preset proportion, the user's viewing angle can be adjusted so that the proportion of the content displayed in the target screen within the display window is increased to a value greater than or equal to the aforementioned preset proportion under the adjusted viewing angle. This avoids the situation where the proportion of the content displayed in the target screen within the display window is too small, resulting in too much white space and making it difficult for the user to observe.

[0116] In some embodiments, when the automatically triggered command includes takeoff or landing commands, the user's viewing angle can be adjusted to switch to a top-down viewing angle. The top-down viewing angle provides a wider field of view, allowing the user to clearly see the environment around the aircraft 110 and the ground conditions, enhancing their overall understanding of the scene.

[0117] In some embodiments, when the automatically triggered command includes an acceleration command, the user's viewing angle can be adjusted to switch to a distant viewing angle. A distant viewing angle helps the user see the surrounding terrain, obstacles, and other dynamic elements more clearly, aids in understanding the relative positions and distances between objects, helps assess the impact and necessity of acceleration, and thus helps in making more informed decisions.

[0118] In some embodiments, when the automatic trigger command includes a deceleration command, the user's viewing angle can be adjusted to switch to a close-up viewing angle. A close-up view allows the user to clearly see details ahead, such as ground conditions, obstacles, and other important information, helping to better determine the necessity and safety of deceleration.

[0119] In some embodiments, when the automatic triggering command includes a turning command, the user's viewing angle can be adjusted to switch to a viewing angle biased towards the turning side. For example, when aircraft 110 turns left, the user's viewing angle can be adjusted to face the left side of aircraft 110; when aircraft 110 turns right, the user's viewing angle can be adjusted to face the right side of aircraft 110. In this way, the user can better observe obstacles affecting the turning of aircraft 110, thereby better controlling the turning process of aircraft 110.

[0120] In some embodiments, the user's viewing angle can be continuously adjusted. For example, when a user adjusts their viewing angle from a top-down view to a bottom-up view, the viewing angle changes by nearly 180 degrees. This change can be continuous and smooth, making the angle change natural, reducing abruptness during the adjustment process, and providing a more comfortable viewing experience. For example, the user can achieve smooth continuous adjustment using a dial. Alternatively, it can be divided into several discrete levels, such as switching directly from a bottom-up view to a top-down view without passing through a level view, thereby improving the speed of switching.

[0121] In some embodiments, the user interface has at least two display windows, including a first window and a second window. The first window and the second window can be presented in a superimposed, picture-in-picture, side-by-side, or other manner.

[0122] Optionally, the first and second windows are used to display different user viewing angles of the target image. Referring to Figure 10A, the first window displays a level viewing angle, and the second window displays a low-angle viewing angle. Referring to Figure 10B, the first window displays a top-down viewing angle, and the second window displays a low-angle viewing angle. In other examples, the first and second windows can also be used to display other types of user viewing angles; in addition to the first and second windows, the aforementioned at least two display windows can include more windows, and the user viewing angles displayed in each window can be fixed or dynamically set according to actual conditions, which will not be elaborated here. Furthermore, although the first and second windows shown in Figures 10A and 10B are displayed in a horizontally tiled manner, and the size of the first window is basically the same as the size of the second window, in practical applications, the display method is not limited to this. For example, the first window can be smaller than the second window, and the first window can be superimposed on the second window (e.g., superimposed on the lower left corner of the second window). In the presence of three or more windows, two or more smaller windows can be superimposed on a larger window (for example, a smaller first window and a smaller second window can be superimposed on the lower left and lower right corners of a larger third window, respectively).

[0123] Optionally, the first window and the second window are used to display images of the target image at different scaling ratios. For example, the first window is used to display the target image at its original size, while the second window is used to display an image obtained by scaling down the original target image by 50%. As another example, the first window is used to display an image obtained by scaling down the original target image by 50%, while the second window is used to display an image obtained by scaling up the original target image by 50%.

[0124] Optionally, the first window and the second window are used to display partial views of different cropped portions of the target image. For example, the first window may display a partial view cropped from the lower left corner of the target image, and the second window may display a partial view cropped from the upper right corner of the target image. Alternatively, the first window may display a partial view cropped from the center area of ​​the target image, and the second window may display a partial view cropped from the upper right corner of the target image. Yet another example is that the first window may display a partial view of an obstacle cropped from the target image, and the second window may display a partial view of another obstacle cropped from the target image.

[0125] Optionally, the first window is used to display the target image, and the second window is used to display other images, which are different from the target image. For example, referring to Figure 11A, the first window is used to display the target image, and the second window is used to display the image transmission image. As another example, referring to Figure 11B, the first window is used to display the target image, and the second window is used to display the radar image.

[0126] In some embodiments, the proportion, position, and / or content displayed by the first and second windows on the user interface can be adjusted. These adjustments may be made in response to user commands or automatically based on sensor detection information or other conditions.

[0127] In some embodiments, the occupancy ratio, position, and / or content displayed by the first window and the second window on the user interface can be switched. For example, assuming that at the current moment, the first window is the smaller window on the user interface and the second window is the larger window, the first window can be switched to the larger window and the second window to the smaller window. As another example, assuming that at the current moment, the first window is the left-hand window on the user interface and the second window is the right-hand window, the first window can be switched to the right-hand window and the second window to the left-hand window. Yet another example, assuming that at the current moment, the first window is used to display the target image and the second window is used to display the radar chart, the first window can be switched to display the radar chart and the second window to display the target image.

[0128] Figure 12 shows the overall system block diagram of this application. The system includes an aircraft 110 and a control terminal 140. The aircraft 110 is an unmanned aerial vehicle (UAV) and includes a flight control system 160, a media center 191, a graphics processor 192, and an encoder 193. The control terminal 140 can be a remote controller. The overall process is as follows:

[0129] a) The graphics processor 192 inside the unmanned aerial vehicle chip receives the point cloud data of the lidar (i.e., the feature point information in the aforementioned embodiment), renders and generates a point cloud image (i.e., the target image in the aforementioned embodiment), and transmits it to the ground remote controller for display through the image transmission channel.

[0130] b) The flight control system 160 calculates the pose information of the aircraft 110 and identifies obstacle feature points (referred to as obstacle points) and non-obstacle feature points (referred to as non-obstacle points) through a semantic recognition algorithm. The media center 191 downsamples the background points (i.e., non-obstacle feature points), such as retaining only one non-obstacle feature point within a 30cm×30cm×30cm space voxel, adjusting the coordinates of this point to the center of the voxel to optimize the display effect. In the vertical direction of the voxel, only the first received point is retained, and all subsequent points are discarded to reduce background point layering and expand the background point coverage area. Similarly, obstacle feature points are also downsampled, such as retaining only one obstacle feature point within a 6cm×6cm×6cm space voxel. The processing method is the same as for non-obstacle feature points. The media center 191 can also calculate the pose of the third-person perspective (i.e., the user's observation perspective), collect the historical flight trajectory of the aircraft 110, and calculate the current danger zone. This information is used by the subsequent image processor 192 for rendering and coloring of various information. The graphics processor 192 colors obstacle points, non-obstacle points, and danger zones, renders virtual markers and virtual trajectories, and outputs third-person view image data (i.e., the target screen). The total number of feature points to be rendered by the graphics processor can be preset, and this total number can be allocated proportionally to obstacle and non-obstacle feature points, serving as the upper limits for the number of obstacle and non-obstacle feature points that can be rendered, respectively. During rendering, different coloring strategies are used for obstacle and non-obstacle feature points; a height-based coloring strategy is used for background points, and a distance-based coloring strategy is used for obstacle points, making it easier for users to identify obstacles in the target screen. The encoder 193 encodes the image data into a bitstream and sends it to the user interface of the control terminal 140 for display.

[0131] c) Mark the area extending a preset distance (e.g., 20 meters) from the UAV's cross-section in the direction of the UAV's flight path as a danger zone. Highlight the feature points in this area to alert the user that a collision may occur.

[0132] d) The point cloud image adopts a third-person perspective: the image contains a simulation model of the UAV that displays its flight attitude in real time and the virtual trajectory of the UAV.

[0133] e) Supports users to adjust the third-person perspective using physical or virtual controls such as dials on the remote control, such as switching between upward, downward, and eye-level perspectives.

[0134] Currently, assisted control technologies in related fields rely on FPV (Field-Propelled Vehicle) units such as unmanned aerial vehicles (UAVs) to assist in operations, which has many limitations. The embodiments of this application, compared to FPV-assisted operations, offer at least the following technical advantages:

[0135] (1) The FPV perspective can only observe the terrain in front. In complex operation scenarios such as woodlands and caves, the work unit is at risk of collision and overturning in all directions, and it is difficult for the user to take into account the situation in all directions. This application generates a third-person target image, which can comprehensively show the risks of collision and overturning of the work unit in all directions, thereby improving safety.

[0136] (2) Most FPVs capture images using optical sensors, which have several limitations at night: I) Visible light is unavailable at night; II) If infrared sensors are used, it is difficult to distinguish backgrounds with similar temperatures. In solutions relying on infrared supplemental lighting, the work unit needs to be very close to the object for it to be clearly distinguishable, which is dangerous for high-speed work units such as UAVs. This application uses sensors such as lidar to collect point cloud data, which is not affected by the intensity of ambient light or the temperature distribution of the object, and does not require the work unit to be close to the object.

[0137] (3) The FPV viewpoint is consistent with the work unit, which means that in situations such as high-speed movement, vibration, and strong wind, the preview image will shake along with the work unit, making it inconvenient to observe. This application obtains the target image based on feature point information. Since downsampling and averaging are usually performed during the generation of the target image, the impact of work unit shaking on observation can be effectively reduced.

[0138] (4) The FPV screen is a two-dimensional screen, and the content is not intuitive. It is relatively difficult to judge the relative positional relationship between scene objects and work units. It requires users to rely on experience to judge, which tests the user's skill level and is unreliable at times, which can easily lead to flight anxiety. The target screen of this application is a three-dimensional screen, which can display the height change of virtual markers relative to the ground feature points in real time; and / or the number of feature points displayed in the target screen changes with the real-time altitude information of the aircraft. It can also further formulate various strategies such as distance rendering strategy and altitude rendering strategy to render feature points at different distances or altitudes. Users can also switch the user's viewing perspective through the control terminal, so that users can intuitively judge the relative positional relationship between scene objects and work units, which can effectively reduce flight anxiety and provide information input for the operator's next flight operation.

[0139] (5) FPV does not have memory capabilities and can only present the currently captured content, making it inconvenient to present historical operational environment records and thus provide a reference for subsequent operations. In this application, the historically collected feature point information can be cached and used together with the currently collected feature point information to generate the target image. At the same time, the target image can also present historical information such as historical trajectories, which is convenient for presenting historical operational environment records.

[0140] It should be noted that the control methods corresponding to the above embodiments can be executed entirely on the aircraft side, entirely on the aircraft's control terminal side, or partially on the aircraft side and partially on the aircraft's control terminal side. This application does not impose any limitations on this.

[0141] In some embodiments where the control method steps are executed on the aircraft side and some steps are executed on the aircraft's control terminal side, in some embodiments, the aircraft's control terminal side executes the following steps independently:

[0142] Referring to Figure 13, this application also provides a display method, including:

[0143] Step S21: During the operation of the aircraft 110, acquire the target image and the virtual identifier superimposed on the target image;

[0144] Step S22: Display the target screen and the virtual logo superimposed on the target screen in real time in the user interface;

[0145] The target image uses feature points to represent the physical environment in which the aircraft 110 is located. The virtual identifier is generated based on the target information of the aircraft 110, which includes the real-time position information of the aircraft 110 during operation. The display position of the virtual identifier on the target image can indicate the relative position of the aircraft 110 in its physical environment. The display position of the virtual identifier on the target image changes as the real-time position information of the aircraft 110 changes.

[0146] The target image can display the real-time changes in the altitude of the virtual marker relative to the ground feature points; and / or the number of feature points displayed in the target image changes with the real-time altitude information of the aircraft 110.

[0147] This embodiment can be executed on the control terminal 140 shown in Figure 1. The control terminal 140 can acquire the target image and the virtual identifier superimposed on the target image from the aircraft 110 corresponding to the control method shown in Figure 14, and display the target image and the virtual identifier superimposed on the target image in real time in the user interface. For specific implementation details of this embodiment, please refer to the foregoing embodiments, which will not be repeated here.

[0148] When some steps of the control method corresponding to the above embodiments are executed by the aircraft and some steps are executed by the aircraft's control terminal, in some embodiments, the aircraft executes the following steps alone:

[0149] Referring to Figure 14, this application also provides a control method, including:

[0150] Step S31: During the operation of the aircraft 110, acquire feature point information of the physical environment in which the aircraft is located;

[0151] Step S32: Based on feature point information, generate a target image for display, wherein the target image uses feature points to characterize the physical environment in which the aircraft 110 is located;

[0152] Step S33: Obtain target information during the operation of the aircraft 110, wherein the target information includes the real-time position information of the aircraft 110;

[0153] Step S34: Based on the target information of the aircraft 110, generate a virtual identifier for display. The display position of the virtual identifier on the target screen can indicate the relative position of the aircraft in its physical environment. The display position of the virtual identifier on the target screen changes with the real-time position information of the aircraft 110; and

[0154] Step S35: Send the virtual identifier and target image to the control terminal 140 of the aircraft 110 so that the user interface of the control terminal 140 can display the target image and the virtual identifier superimposed on the target image.

[0155] The target image can display the real-time changes in the altitude of the virtual marker relative to the ground feature points; and / or the number of feature points displayed in the target image changes with the real-time altitude information of the aircraft 110.

[0156] This embodiment can be executed on the aircraft 110 side shown in Figure 1. The aircraft 110 can acquire the target image and generate a virtual identifier, and send the target image and virtual identifier to the control terminal 140 side corresponding to the display mode shown in Figure 13, so that the user interface of the control terminal 140 can display the target image and the virtual identifier superimposed on the target image. For the specific implementation details of this embodiment, please refer to the foregoing embodiments, which will not be repeated here.

[0157] To facilitate users' intuitive understanding of the relative layout changes between detected and undetected areas in a third-person view, this application also proposes the following solution.

[0158] Referring to Figure 15, this application also provides a control method, including:

[0159] Step S41: Use the detection signals actively sent by the sensors of the aircraft 110 to obtain the feature point information of the physical environment in which the aircraft 110 is located.

[0160] Step S42: Based on feature point information, generate a target image for display, wherein the target image uses feature points to characterize the physical environment in which the aircraft 110 is located;

[0161] Step S43: Obtain target information during the operation of the aircraft 110, wherein the target information includes the real-time position information of the aircraft 110;

[0162] Step S44: Based on the target information of the aircraft 110, generate a virtual identifier for display. The display position of the virtual identifier on the target screen indicates the relative position of the aircraft in its physical environment. The display position of the virtual identifier on the target screen changes with the real-time position information of the aircraft 110; and

[0163] Step S45: Display the target screen and the virtual logo superimposed on the target screen on the user interface;

[0164] The target image includes a first region that has been detected and a second region that has not been detected. The relative layout of the first and second regions changes with the real-time altitude information of the aircraft 110.

[0165] In step S41, sensors on the aircraft 110 can actively transmit detection signals. By transmitting detection signals, feature point information of the physical environment in which the aircraft 110 is located can be obtained. For example, the sensor is a lidar, which can actively transmit laser signals to obtain point cloud information of the physical environment in which the aircraft 110 is located.

[0166] The specific implementation methods of steps S42 to S45 can be referred to the aforementioned embodiments, and will not be repeated here.

[0167] It should be noted that, due to the limited detection range of the sensor, the target image may include a first region that has been detected and a second region that has not been detected. For example, in the example shown in Figure 4, the black area is the second region that has not been detected, and the area outside the black area is the first region that has been detected.

[0168] The relative arrangement of the first and second regions changes with the real-time altitude information of the aircraft 110. For example, as the real-time altitude information of the aircraft 110 increases, the sensing range of the sensors also increases, thus increasing the coverage area of ​​the first region in the target image and decreasing the coverage area of ​​the second region. Similarly, as the real-time altitude information of the aircraft 110 decreases, the sensing range of the sensors also decreases, thus decreasing the coverage area of ​​the first region in the target image and increasing the coverage area of ​​the second region.

[0169] In this embodiment, a third-person target view is generated in real time during the operation of the aircraft. Feature points are used to characterize the physical environment in which the aircraft is located. Virtual markers are generated in real time to indicate the relative position of the aircraft within its physical environment. The target view and the virtual markers superimposed on it are displayed on the user interface. The target view is generated using actively detected information, allowing users to intuitively perceive the relative position of the aircraft in the physical environment and potential safety hazards in multiple directions, reducing the difficulty of flight operation. The target view includes a first area detected by the aircraft and a second area not detected by the aircraft. The relative layout of the first and second areas changes with the real-time altitude information of the aircraft, enabling users to more intuitively observe changes in detected information due to changes in the aircraft's flight altitude, thus improving the user experience.

[0170] It should be noted that the control methods corresponding to the above embodiments can be executed entirely on the aircraft side, entirely on the aircraft's control terminal side, or partially on the aircraft side and partially on the aircraft's control terminal side. This application does not impose any limitations on this.

[0171] In some embodiments where the control method steps are executed on the aircraft side and some steps are executed on the aircraft's control terminal side, in some embodiments, the aircraft's control terminal side executes the following steps independently:

[0172] Referring to Figure 16, this application also provides a display method, including:

[0173] Step S51: During the operation of the aircraft 110, acquire the target image and the virtual identifier superimposed on the target image;

[0174] Step S52: Display the target screen and the virtual logo superimposed on the target screen in real time in the user interface;

[0175] The target image uses feature points to represent the physical environment in which the aircraft 110 is located. These feature points are acquired based on detection signals actively transmitted by the aircraft's sensors. The virtual identifier is generated based on the target information of the aircraft 110, which includes the real-time position information of the aircraft 110 during its operation. The display position of the virtual identifier on the target image indicates the relative position of the aircraft in its physical environment, and the display position of the virtual identifier on the target image changes as the real-time position information of the aircraft changes.

[0176] The target image includes a first region that has been detected and a second region that has not been detected. The relative layout of the first and second regions changes with the real-time altitude information of the aircraft 110.

[0177] This embodiment can be executed on the control terminal 140 shown in Figure 1. The control terminal 140 can acquire the target image and the virtual identifier superimposed on the target image from the aircraft 110 corresponding to the control method shown in Figure 17, and display the target image and the virtual identifier superimposed on the target image in real time in the user interface. For specific implementation details of this embodiment, please refer to the foregoing embodiments, which will not be repeated here.

[0178] In some embodiments where the control method steps are executed on the aircraft side and some steps are executed on the aircraft's control terminal side, in some embodiments, the following steps are executed separately on the aircraft side:

[0179] Referring to Figure 17, this application also provides a control method, including:

[0180] Step S61: Use the detection signals actively sent by the sensors of the aircraft 110 to obtain the feature point information of the physical environment in which the aircraft 110 is located.

[0181] Step S62: Based on feature point information, generate a target image for display, wherein the target image uses feature points to characterize the physical environment in which the aircraft 110 is located;

[0182] Step S63: Obtain target information during the operation of the aircraft 110, wherein the target information includes the real-time position information of the aircraft 110;

[0183] Step S64: Based on the target information of the aircraft 110, generate a virtual identifier for display. The display position of the virtual identifier on the target screen can indicate the relative position of the aircraft 110 in its physical environment. The display position of the virtual identifier on the target screen changes with the real-time position information of the aircraft 110; and

[0184] Step S65: Send the virtual identifier and target image to the control terminal 140 of the aircraft 110 so that the user interface of the control terminal 140 can display the target image and the virtual identifier superimposed on the target image.

[0185] The target image includes a first region that has been detected and a second region that has not been detected. The relative layout of the first and second regions changes with the real-time altitude information of the aircraft 110.

[0186] This embodiment can be executed on the aircraft 110 shown in Figure 1. The aircraft 110 can acquire the target image and generate a virtual identifier, and send the target image and virtual identifier to the control terminal 140 corresponding to the display method shown in Figure 16, so that the user interface of the control terminal 140 can display the target image and the virtual identifier superimposed on the target image. For specific implementation details of this embodiment, please refer to the foregoing embodiments, which will not be repeated here.

[0187] To better assist users in navigating narrow passages (such as through caves, tunnels, or underpasses) or in situations with overhead obstacles, this application also provides the following solutions.

[0188] Referring to Figure 18, this application also provides a control method, including:

[0189] Step S71: During the operation of the aircraft 110, acquire feature point information of the physical environment in which the aircraft 110 is located;

[0190] Step S72: Based on feature point information, generate a target image for display, wherein the target image uses feature points to characterize the physical environment in which the aircraft 110 is located;

[0191] Step S73: Obtain target information during the operation of the aircraft 110, wherein the target information includes the real-time position information of the aircraft 110;

[0192] Step S74: Based on the target information of the aircraft 110, generate a virtual identifier for display. The display position of the virtual identifier on the target screen indicates the relative position of the aircraft 110 in its physical environment. The display position of the virtual identifier on the target screen changes with the real-time position information of the aircraft 110; and

[0193] Step S75: Display the target screen and the virtual logo superimposed on the target screen on the user interface;

[0194] Among them, the user's viewing angle of the target screen includes an upward viewing angle, in which the user can observe the bottom of the virtual logo.

[0195] The specific implementation details of this embodiment are detailed in the foregoing embodiments and will not be repeated here. It is particularly important to note that the user's viewing angle of the target image in this application includes a downward-looking viewing angle, as shown in Figures 9, 10A, and 10B. In related technologies, users typically only obtain a first-person view and operate the aircraft 110 based on this. However, when the aircraft 110 passes through narrow passages such as underpasses, there may be a collision risk in areas such as the top or sides of the aircraft 110. By adopting a downward-looking viewing angle, the aircraft 110 can be observed from below, thereby facilitating the elimination of risk factors in areas such as the top or sides of the aircraft 110 and improving flight safety.

[0196] This application generates a third-person target image in real time during the operation of the aircraft. It uses feature points to characterize the physical environment in which the aircraft is located and generates virtual markers in real time to indicate the relative position of the aircraft in its physical environment. The target image and the virtual markers superimposed on the target image are displayed on the user interface. This allows users to intuitively perceive the relative position of the aircraft in the physical environment and the potential safety hazards in multiple directions, reducing the difficulty of operating the aircraft. In addition, the viewing angle of the target image includes the user's upward viewing angle, which helps the user observe the relative positional relationship between the aircraft and the environmental objects above it. This helps the user operate the aircraft more safely when passing through narrow passages (such as going through caves, tunnels, or under bridges) or in scenarios with overhead obstacles (such as flying under a utility pole or under an indoor ceiling), avoiding contact with overhead obstacles.

[0197] It should be noted that the control methods corresponding to the above embodiments can be executed entirely on the aircraft side, entirely on the aircraft's control terminal side, or partially on the aircraft side and partially on the aircraft's control terminal side. This application does not impose any limitations on this.

[0198] In some embodiments where the control method steps are executed on the aircraft side and some steps are executed on the aircraft's control terminal side, in some embodiments, the aircraft's control terminal side executes the following steps independently:

[0199] Referring to Figure 19, this application also provides a display method, including:

[0200] Step S81: During the operation of the aircraft 110, acquire the target image and the virtual identifier superimposed on the target image;

[0201] Step S82: Display the target screen and the virtual logo superimposed on the target screen in real time in the user interface;

[0202] The target screen uses feature points to represent the physical environment in which the aircraft 110 is located. The virtual identifier is generated based on the target information of the aircraft 110. The target information includes the real-time position information of the aircraft 110 during operation. The display position of the virtual identifier on the target screen can indicate the relative position of the aircraft 110 in its physical environment. The display position of the virtual identifier on the target screen changes with the real-time position information of the aircraft. The user's viewing angle of the target screen includes an upward viewing angle. In the upward viewing angle, the user can observe the bottom surface of the virtual identifier.

[0203] This embodiment can be executed on the control terminal 140 shown in Figure 1. The control terminal 140 can acquire the target image and the virtual identifier superimposed on the target image from the aircraft 110 corresponding to the control method shown in Figure 20, and display the target image and the virtual identifier superimposed on the target image in real time in the user interface. For specific implementation details of this embodiment, please refer to the foregoing embodiments, which will not be repeated here.

[0204] In some embodiments where the control method steps are executed on the aircraft side and some steps are executed on the aircraft's control terminal side, the following steps are executed separately on the aircraft side in some embodiments:

[0205] Referring to Figure 20, this application also provides a control method, including:

[0206] Step S91: During the operation of the aircraft 110, acquire feature point information of the physical environment in which the aircraft 110 is located;

[0207] Step S92: Based on feature point information, generate a target image for display, wherein the target image uses feature points to characterize the physical environment in which the aircraft 110 is located;

[0208] Step S93: Obtain target information during the operation of the aircraft 110, wherein the target information includes the real-time position information of the aircraft 110;

[0209] Step S94: Based on the target information of the aircraft 110, generate a virtual identifier for display. The display position of the virtual identifier on the target screen indicates the relative position of the aircraft 110 in its physical environment. The display position of the virtual identifier on the target screen changes with the real-time position information of the aircraft 110; and

[0210] Step S95: Send the virtual identifier and target image to the control terminal of the aircraft 110 so that the user interface of the control terminal can display the target image and the virtual identifier superimposed on the target image.

[0211] Among them, the user's viewing angle of the target screen includes an upward viewing angle, in which the user can observe the bottom of the virtual logo.

[0212] This embodiment can be executed on the aircraft 110 shown in Figure 1. The aircraft 110 can acquire the target image and generate a virtual identifier, and send the target image and virtual identifier to the control terminal 140 corresponding to the display method shown in Figure 19, so that the user interface of the control terminal 140 can display the target image and the virtual identifier superimposed on the target image. For detailed implementation of this embodiment, please refer to the foregoing embodiments, which will not be repeated here.

[0213] This application also provides an aircraft, including a memory and a processor;

[0214] The memory is used to store computer programs;

[0215] The processor is configured to execute the computer program and, when executing the computer program, implement the control method as described above.

[0216] This application also provides a control terminal, including a memory and a processor;

[0217] The memory is used to store computer programs;

[0218] The processor is configured to execute the computer program and, when executing the computer program, implement the display method as described above.

[0219] This application also provides a system, including:

[0220] At least one processor; and

[0221] At least one memory including computer program code, wherein the at least one memory and the computer program code are configured together with at least one processor to enable the system to perform at least the methods described in any embodiment of the present application.

[0222] The system in this application embodiment includes an aircraft and a control terminal. The aircraft can be the aircraft described above, and the control terminal can be the control terminal described above.

[0223] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the methods described in any of the foregoing embodiments.

[0224] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0225] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that the embodiments of this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions of the embodiments of this application, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0226] The systems, devices, modules, or units described in the above embodiments can be implemented by computer devices or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0227] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate. When implementing the embodiments of this application, the functions of each module can be implemented in one or more software and / or hardware. Alternatively, some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0228] The above description is only a specific implementation of the embodiments of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the embodiments of this application, and these improvements and modifications should also be considered as the protection scope of the embodiments of this application.

Claims

1. A control method, characterized in that, include: During the operation of the aircraft, a detection beam is actively emitted to obtain feature point information of the physical environment in which the aircraft is located; Based on the feature point information, a target image for display is generated, wherein the target image uses feature points to characterize the physical environment in which the aircraft is located; Acquire target information during the operation of the aircraft, wherein the target information includes the real-time position information of the aircraft; Based on the target information of the aircraft, a virtual identifier is generated for display. The display position of the virtual identifier on the target screen indicates the relative position of the aircraft in its physical environment. The display position of the virtual identifier on the target screen changes with the real-time position information of the aircraft. The target screen and the virtual icon superimposed on the target screen are displayed on the user interface; The target screen can display the real-time height changes of the virtual marker relative to the ground feature points; and / or the number of feature points displayed in the target screen changes with the real-time altitude information of the aircraft.

2. A display method, characterized in that, include: During the operation of the aircraft, the target image and the virtual identifier superimposed on the target image are acquired; The target screen and the virtual icon superimposed on the target screen are displayed in real time in the user interface; The target image uses feature points to represent the physical environment in which the aircraft is located. These feature points are acquired based on detection signals actively transmitted by the aircraft's sensors. The virtual identifier is generated based on the aircraft's target information, which includes the aircraft's real-time position information during operation. The display position of the virtual identifier on the target image indicates the aircraft's relative position within its physical environment, and this position changes as the aircraft's real-time position information changes. The target screen can display the real-time height changes of the virtual marker relative to the ground feature points; and / or the number of feature points displayed in the target screen changes with the real-time altitude information of the aircraft.

3. A control method, characterized in that, include: During the operation of the aircraft, a detection beam is actively emitted to obtain feature point information of the physical environment in which the aircraft is located; Based on the feature point information, a target image for display is generated, wherein the target image uses feature points to characterize the physical environment in which the aircraft is located; Acquire target information during the operation of the aircraft, wherein the target information includes the real-time position information of the aircraft; Based on the target information of the aircraft, a virtual identifier is generated for display. The display position of the virtual identifier on the target screen indicates the relative position of the aircraft in its physical environment. The display position of the virtual identifier on the target screen changes with the real-time position information of the aircraft. The virtual identifier and the target image are sent to the control terminal of the aircraft so that the user interface of the control terminal can display the target image and the virtual identifier superimposed on the target image; The target screen can display the real-time height changes of the virtual marker relative to the ground feature points; and / or the number of feature points displayed in the target screen changes with the real-time altitude information of the aircraft.

4. A control method, characterized in that, include: The aircraft's sensors actively transmit detection signals to obtain feature point information of the physical environment in which the aircraft is located; Based on the feature point information, a target image for display is generated, wherein the target image uses feature points to represent the target image. Describe the physical environment in which the aircraft is located; Acquire target information during the operation of the aircraft, wherein the target information includes the real-time position information of the aircraft; Based on the target information of the aircraft, a virtual identifier is generated for display. The display position of the virtual identifier on the target screen indicates the relative position of the aircraft in its physical environment. The display position of the virtual identifier on the target screen changes with the real-time position information of the aircraft. The target screen and the virtual icon superimposed on the target screen are displayed on the user interface; The target image includes a first region that has been detected and a second region that has not been detected. The relative layout of the first region and the second region changes with the real-time altitude information of the aircraft.

5. A display method, characterized in that, include: During the operation of the aircraft, the target image and the virtual identifier superimposed on the target image are acquired; The target screen and the virtual icon superimposed on the target screen are displayed in real time in the user interface; The target image uses feature points to represent the physical environment in which the aircraft is located. These feature points are acquired based on detection signals actively transmitted by the aircraft's sensors. The virtual identifier is generated based on the aircraft's target information, which includes the aircraft's real-time position information during operation. The display position of the virtual identifier on the target image indicates the aircraft's relative position within its physical environment, and this position changes as the aircraft's real-time position information changes. The target image includes a first region that has been detected and a second region that has not been detected. The relative layout of the first region and the second region changes with the real-time altitude information of the aircraft.

6. A control method, characterized in that, include: The aircraft's sensors actively transmit detection signals to obtain feature point information of the physical environment in which the aircraft is located; Based on the feature point information, a target image for display is generated, wherein the target image uses feature points to characterize the physical environment in which the aircraft is located; Acquire target information during the operation of the aircraft, wherein the target information includes the real-time position information of the aircraft; Based on the target information of the aircraft, a virtual identifier is generated for display. The display position of the virtual identifier on the target screen indicates the relative position of the aircraft in its physical environment. The display position of the virtual identifier on the target screen changes with the real-time position information of the aircraft. The virtual identifier and the target image are sent to the control terminal of the aircraft so that the user interface of the control terminal can display the target image and the virtual identifier superimposed on the target image; The target image includes a first region that has been detected and a second region that has not been detected. The relative layout of the first region and the second region changes with the real-time altitude information of the aircraft.

7. A control method, characterized in that, include: During the operation of the aircraft, a detection beam is actively emitted to obtain feature point information of the physical environment in which the aircraft is located; Based on the feature point information, a target image for display is generated, wherein the target image uses feature points to characterize the physical environment in which the aircraft is located; Acquire target information during the operation of the aircraft, wherein the target information includes the real-time position information of the aircraft; Based on the target information of the aircraft, a virtual identifier is generated for display. The display position of the virtual identifier on the target screen indicates the relative position of the aircraft in its physical environment. The displayed position changes with the real-time position information of the aircraft; and The target screen and the virtual icon superimposed on the target screen are displayed on the user interface; The user's viewing angle of the target image includes a downward viewing angle, under which the user can observe the bottom surface of the virtual icon.

8. A display method, characterized in that, include: During the operation of the aircraft, the target image and the virtual identifier superimposed on the target image are acquired; The target screen and the virtual icon superimposed on the target screen are displayed in real time in the user interface; The target image uses feature points to characterize the physical environment in which the aircraft is located. These feature points are obtained based on detection signals actively sent by the aircraft's sensors. The virtual identifier is generated based on the aircraft's target information, which includes the aircraft's real-time position information during operation. The display position of the virtual identifier on the target image indicates the relative position of the aircraft in its physical environment. The display position of the virtual identifier on the target image changes with the aircraft's real-time position information. The user's viewing angle of the target image includes an upward viewing angle, under which the user can observe the bottom surface of the virtual identifier.

9. A control method, characterized in that, include: During the operation of the aircraft, a detection beam is actively emitted to obtain feature point information of the physical environment in which the aircraft is located; Based on the feature point information, a target image for display is generated, wherein the target image uses feature points to characterize the physical environment in which the aircraft is located; Acquire target information during the operation of the aircraft, wherein the target information includes the real-time position information of the aircraft; Based on the target information of the aircraft, a virtual identifier is generated for display. The display position of the virtual identifier on the target screen indicates the relative position of the aircraft in its physical environment. The display position of the virtual identifier on the target screen changes with the real-time position information of the aircraft. The virtual identifier and the target image are sent to the control terminal of the aircraft so that the user interface of the control terminal can display the target image and the virtual identifier superimposed on the target image; The user's viewing angle of the target image includes a downward viewing angle, under which the user can observe the bottom surface of the virtual icon.

10. The method according to any one of claims 1 to 9, characterized in that, The target information also includes the real-time attitude information of the aircraft, and the attitude of the virtual identifier in the target image changes as the real-time attitude information of the aircraft changes.

11. The method according to claim 10, characterized in that, The real-time attitude information of the aircraft is obtained by the aircraft's sensors in real time.

12. The method according to any one of claims 1 to 9, characterized in that, The target information also includes the real-time orientation information of the aircraft, and the orientation of the virtual icon in the target image changes as the real-time orientation information of the aircraft changes.

13. The method according to claim 12, characterized in that, The real-time orientation information of the aircraft is obtained by the aircraft's sensors in real time.

14. The method according to any one of claims 1 to 9, characterized in that, The virtual identifier includes a simulation model of the aircraft.

15. The method according to claim 14, characterized in that, The rendering parameters of the simulation model correspond to the characteristics of the physical entity of the aircraft.

16. The method according to claim 15, characterized in that, The simulation model is obtained by scaling the physical entity of the aircraft to a preset scale.

17. The method according to claim 16, characterized in that, The size ratio between the simulation model and the physical entity of the aircraft is the same as the size ratio between the object displayed in the target image and its corresponding entity in the physical environment.

18. The method according to claim 15, characterized in that, The physical entity's characteristics include its shape.

19. The method according to claim 18, characterized in that, The external features include: outline shape or size.

20. The method according to any one of claims 1 to 9, characterized in that, The method further includes: overlaying a virtual trajectory of the aircraft onto the target image, the virtual trajectory indicating the relative position of the aircraft's motion trajectory in the physical environment in which the aircraft is located.

21. The method according to claim 20, characterized in that, The virtual trajectory changes as the motion trajectory changes.

22. The method according to claim 20, characterized in that, The motion trajectory includes historical motion trajectories.

23. The method according to claim 22, characterized in that, The historical trajectory includes the historical trajectory of the aircraft tracing back a preset time from the current moment.

24. The method according to claim 23, characterized in that, The preset duration is 1 to 3 seconds.

25. The method according to claim 20, characterized in that, The motion trajectory includes the planned motion trajectory.

26. The method according to claim 25, characterized in that, The planned motion trajectory includes the planned motion trajectory of the aircraft from the current moment forward for a preset time period.

27. The method according to claim 20, characterized in that, The step of overlaying the virtual trajectory of the aircraft onto the target screen includes: overlaying the virtual trajectory of the aircraft onto the target screen in a preset style.

28. The method according to claim 27, characterized in that, The virtual trajectory extends radially from the virtual identifier.

29. The method according to claim 27, characterized in that, The virtual trajectory includes trajectory elements, which include trajectory points and / or trajectory segments.

30. The method according to claim 29, characterized in that, The trajectory elements include specific trajectory elements, and the display style of the specific trajectory elements is different from the display style of other trajectory elements besides the specific trajectory elements.

31. The method according to claim 30, characterized in that, The specific trajectory elements are used to indicate at least one of the following: motion risk point, motion start point, motion end point, motion turning point, motion interruption point, motion continuation point, task point, and non-task point.

32. The method according to any one of claims 1 to 9, characterized in that, The method further includes: displaying specific display elements on the target screen using a preset style.

33. The method according to claim 32, characterized in that, The specific display elements are used to indicate the operating area of ​​the aircraft.

34. The method according to claim 32, characterized in that, The specific display elements are used to indicate specific trajectory points of the aircraft.

35. The method according to claim 34, characterized in that, The specific trajectory points include any of the following: motion start point, motion end point, motion turning point, motion interruption point, motion continuation point, task point, and non-task point.

36. The method according to claim 32, characterized in that, The specific display elements are used to indicate content related to the collision risk of the aircraft.

37. The method according to claim 36, characterized in that, The content related to collision risk includes: a spatial region extending a predetermined distance from the aircraft along its forward direction.

38. The method according to claim 37, characterized in that, The preset distance is 10 to 30 meters.

39. The method according to claim 37, characterized in that, The preset distance is positively correlated with the size of the physical entity of the aircraft.

40. The method according to claim 37, characterized in that, The preset distance is positively correlated with the speed of the aircraft.

41. The method according to claim 36, characterized in that, The content related to collision risk includes: potential collision obstacles.

42. The method according to claim 41, characterized in that, The potential collision obstacles include: potential collision obstacles within a spatial region extending a predetermined distance from the aircraft along its forward direction.

43. The method according to claim 36, characterized in that, The content related to collision risk includes: potential collision direction.

44. The method according to claim 36, characterized in that, The content related to collision risk includes: potential collision locations.

45. The method according to claim 44, characterized in that, The potential collision locations include: the potential collision site on the aircraft side, and / or the potential collision site on the obstacle side.

46. ​​The method according to claim 32, characterized in that, The display style of the specific display element in the target image is different from the display style of other feature points in the target image.

47. The method according to claim 46, characterized in that, The specific display elements are highlighted in the target image.

48. The method according to claim 32, characterized in that, The method further includes: performing hidden point removal processing on other feature points in the target image besides the specific display element.

49. The method according to any one of claims 1 to 9, characterized in that, The user's viewing angle of the target image can be adjusted to allow the user to observe the target image from different perspectives.

50. The method according to claim 49, characterized in that, The method further includes: adjusting the user's viewing angle in response to the user's adjustment command.

51. The method according to claim 50, characterized in that, The adjustment commands originate from user input on the aircraft's control terminal.

52. The method according to claim 51, characterized in that, The adjustment command originates from the user's input operation on the preset physical controls of the control terminal.

53. The method according to claim 52, characterized in that, The preset physical control includes a dial, and the adjustment command originates from the user's rotation operation on the dial.

54. The method according to claim 50, characterized in that, The adjustment instructions originate from user input on the user interface.

55. The method according to claim 54, characterized in that, The adjustment instructions originate from the user's editing operations on the preset virtual controls on the user interface.

56. The method according to claim 55, characterized in that, The adjustment instructions originate from the user's editing operations on the virtual identifier on the user interface.

57. The method according to claim 55, characterized in that, The preset virtual control includes at least one of the following: a slider, an input box, a dial, or an icon.

58. The method according to claim 49, characterized in that, The method further includes: adjusting the user's viewing angle in response to an automatic trigger command.

59. The method according to claim 58, characterized in that, The adjustment of the user's viewing perspective includes any one of the following: The user's observation perspective is adjusted so that, compared to the original user's observation perspective, the adjusted user's observation perspective is more conducive to observing factors that affect the maneuvering behavior of the aircraft. The user's viewing angle is adjusted so that the adjusted user viewing angle is more suitable for the proportion of the content displayed in the target screen in the display window compared to the original user viewing angle.

60. The method according to claim 58, characterized in that, The adjustment of the user's viewing perspective includes any one of the following: When the automatic trigger command includes a takeoff or landing command, the user's viewing angle is adjusted to switch to a top-down viewing angle; When the automatic trigger command includes an acceleration command, the user's viewing angle is adjusted to switch to a long-distance viewing angle; When the automatic trigger command includes a deceleration command, the user's viewing angle is adjusted to switch to a close-up viewing angle; When the automatic trigger command includes a turning command, the user's viewing angle is adjusted to switch to a viewing angle biased towards the turning side.

61. The method according to claim 58, characterized in that, The user's viewing angle can be continuously adjusted.

62. The method according to claim 58, characterized in that, The user's viewing angle can be adjusted around a preset axis of the virtual icon, and the preset axis further includes at least one of the yaw axis, pitch axis and roll axis of the virtual icon.

63. The method according to claim 62, characterized in that, When the preset pivot includes the pitch axis of the virtual icon, the user's viewing angle includes at least one of a top-down viewing angle, a level viewing angle, and a bottom-up viewing angle, wherein, in the top-down viewing angle, the user can observe the top surface of the virtual icon; and in the bottom-up viewing angle, the user can observe the bottom surface of the virtual icon.

64. The method according to claim 63, characterized in that, From the top-down viewing angle, the user can at least observe the top surface of the rotor of the virtual icon; from the bottom-up viewing angle, the user can at least observe the bottom surface of the rotor of the virtual icon.

65. The method according to claim 63, characterized in that, The target screen can display in real time the height changes of feature points on the bottom surface of the virtual icon relative to the ground.

66. The method according to claim 63, characterized in that, The head-up viewing angle is the default viewing angle.

67. The method according to any one of claims 1 to 9, characterized in that, The user's viewing angle of the target image can be locked relative to a preset axis of the virtual icon, and the preset axis further includes at least one of the yaw axis, pitch axis and roll axis of the virtual icon.

68. The method according to any one of claims 1 to 9, characterized in that, The step of obtaining feature point information of the physical environment in which the aircraft is located also includes: Based on feature point information, semantic labels for different semantic objects are obtained, including obstacle categories and non-obstacle categories; Based on semantic labels, obstacle feature points and non-obstacle feature points are identified.

69. The method according to claim 68, characterized in that, The obstacle feature points in the target image are rendered using a first rendering strategy, and the non-obstacle feature points in the target image are rendered using a second rendering strategy, wherein the first rendering strategy is different from the second rendering strategy.

70. The method according to claim 69, characterized in that, The first rendering strategy includes a distance rendering strategy, which indicates that feature points at different distances from the virtual identifier are rendered with different colors.

71. The method according to claim 69, characterized in that, The second coloring strategy includes a height rendering strategy, which indicates that feature points at different heights from the virtual identifier are rendered with different colors.

72. The method according to claim 68, characterized in that, The rendering density of obstacle feature points is greater than the rendering density of non-obstacle feature points.

73. The method according to any one of claims 1 to 9, characterized in that, The user interface has at least two display windows, including a first window and a second window.

74. The method according to claim 73, characterized in that, The first window and the second window are used to display the target screen from different user viewing angles.

75. The method according to claim 73, characterized in that, The first window and the second window are used to display the target image at different scaling ratios.

76. The method according to claim 73, characterized in that, The first window and the second window are respectively used to display partial images of different cropped portions of the target image.

77. The method according to claim 73, characterized in that, The first window is used to display the target image, and the second window is used to display other images, which are different from the target image.

78. The method according to claim 73, characterized in that, The proportion of the first window and the second window on the user interface, their position on the user interface, and / or the content they display can be adjusted.

79. The method according to claim 78, characterized in that, The proportion of the first window and the second window on the user interface, their position on the user interface, and / or the content they display can be switched between each other.

80. The method according to any one of claims 4 to 9, characterized in that, The method further includes: in response to a change in the flight altitude of the aircraft, the number of feature points displayed in the target image also changes.

81. The method according to any one of claims 1 to 3, 80, characterized in that, In response to a change in the flight altitude of the aircraft, the number of feature points on the ground in the target image also changes.

82. The method according to claim 81, characterized in that, As the aircraft's flight altitude increases, the number of feature points on the ground in the target image decreases.

83. The method according to claim 81, characterized in that, As the flight altitude of the aircraft decreases, the number of feature points on the ground in the target image increases.

84. The method according to any one of claims 4 to 9, characterized in that, The target screen can display the real-time changes in the height of the virtual icon relative to the feature points on the ground.

85. The method according to any one of claims 1 to 3, 84, characterized in that, The target screen can display in real time the height changes of specific components of the virtual identifier relative to feature points on the ground.

86. The method according to claim 85, characterized in that, The specific component corresponds to the rotor of the aircraft.

87. The method according to claim 85, characterized in that, The specific component corresponds to the payload of the aircraft.

88. The method according to any one of claims 1 to 9, characterized in that, The target image includes a three-dimensional target image.

89. The method according to any one of claims 1 to 9, characterized in that, The feature point information includes: feature point information collected by the aircraft's sensors at the current moment.

90. The method according to claim 89, characterized in that, The feature point information also includes feature point information collected by the aircraft's sensors at previous moments.

91. The method according to claim 89, characterized in that, The aircraft's sensors include lidar.

92. The method according to claim 91, characterized in that, The aircraft's sensors include an omnidirectional lidar.

93. The method according to any one of claims 1 to 9, characterized in that, The target image includes a point cloud image.

94. A computer device, characterized in that, Including memory and processor; The memory is used to store computer programs; The processor is configured to execute the computer program and, when executing the computer program, implement the method of any one of claims 1 to 92.

95. An aircraft, characterized in that, Including memory and processor; The memory is used to store computer programs; The processor is configured to execute the computer program and, in executing the computer program, implement the method according to any one of claims 3, 6, and 9.

96. A control terminal, characterized in that, Including memory and processor; The memory is used to store computer programs; The processor is configured to execute the computer program and, in executing the computer program, implement the method according to any one of claims 2, 5, and 8.

97. A computer-readable storage medium storing computer instructions thereon, characterized in that, When executed by a processor, the computer instructions implement the steps of the method according to any one of claims 1 to 93.

98. A system, characterized in that, include: At least one processor; as well as At least one memory including computer program code, wherein the at least one memory and the computer program code are configured together with at least one processor to enable the system to perform at least the method of any one of claims 1 to 93.

Images