Auxiliary image processing method and device and readable storage medium

Abstract

The invention discloses an auxiliary image processing method and device and a readable storage medium, and the method comprises the steps: obtaining image information collected by a plurality of first image collection devices of an aircraft, the plurality of first image collection devices being used for sensing obstacles around the aircraft; image information collected by a first image collection device used for obstacle avoidance in the current flight speed direction of the aircraft is divided into two paths to be transmitted, and the image information transmitted in one path is used for sensing obstacles around the aircraft, so that the aircraft autonomously avoids the obstacles according to the image information. At least a part of image of the other path of transmitted image information is used as a first image, the first image is used for being watched by a user, and a second image acquisition direction corresponding to the first image is changed along with the change of the flight speed direction; outputting the first image; wherein when the change of the flight speed direction is always within the visual angle range of the same first image acquisition device, the first image is always from the same first image acquisition device.

Description

Auxiliary image processing method, device and readable storage medium Technical Field

[0001] The present application relates to the field of images, and in particular to a method, device and readable storage medium for processing auxiliary images. Background Art

[0002] With the development of technologies such as transportation vehicles and robots, users are increasingly concerned about their operational safety. Vehicles are limited by their steering wheels and drive structure, resulting in relatively fixed steering angles and movement paths. Therefore, they can observe their surroundings through rearview mirrors facing in a specific direction.

[0003] However, robots and multi-rotor aircraft with universal wheels offer flexible and adaptable motion paths and steering angles, enabling parallel and diagonal movement, which is unattainable for vehicles. Using a fixed-angle rearview mirror to observe the surrounding environment can easily lead to inability to observe the surrounding environment due to changes in the direction of movement, potentially causing safety accidents.

[0004] Summary of the Invention

[0005] Based on this, the embodiments of the present application provide a method, device and readable storage medium for processing auxiliary images, aiming to solve the technical problem that the environment in the corresponding direction cannot be observed due to changes in the moving direction during the driving of the movable platform, resulting in low driving safety.

[0006] In a first aspect, an embodiment of the present application provides a method for processing auxiliary images, including: obtaining the current flight speed direction of an aircraft; in response to a situation where the current flight speed direction of the aircraft is inconsistent with the first image acquisition direction of the aircraft, obtaining image information of a first perspective acquired by the image acquisition device of the aircraft in a second image acquisition direction, wherein the second image acquisition direction is associated with the current flight speed direction of the aircraft and is different from the first image acquisition direction; based on the current flight speed direction of the aircraft, determining image information of a second perspective from the image information of the first perspective; and outputting image information of the second perspective, wherein the second perspective is included in the first perspective, and the image information of the second perspective is used to obtain the environmental conditions of the current flight speed direction of the aircraft.

[0007] In a second aspect, an embodiment of the present application provides a method for processing an auxiliary image, comprising:

[0008] Acquire composite image information of a first perspective captured by at least two image acquisition devices of an aircraft; determine image information of a second perspective from the composite image information of the first perspective based on the current flight speed and direction of the aircraft, wherein the second perspective is included in the first perspective; and output the image information of the second perspective, wherein the image information of the second perspective is used to obtain environmental conditions in the current flight speed and direction of the aircraft.

[0009] In a third aspect, an embodiment of the present application provides a method for processing auxiliary images, including: obtaining the current flight speed direction of the aircraft; when the first image acquisition direction of the aircraft is inconsistent with the current flight speed direction, outputting a first image in a second image acquisition direction, wherein the second image acquisition direction is different from the first image acquisition direction, and the second image acquisition direction changes with the change of the flight speed direction of the aircraft.

[0010] In a fourth aspect, an embodiment of the present application provides a method for processing an auxiliary image, comprising:

[0011] Get the current flight speed and direction of the aircraft;

[0012] Based on the current flight speed direction of the aircraft, a first image in a second image acquisition direction and a second image in a first image acquisition direction are output, wherein the second image acquisition direction changes with the change of the flight speed direction of the aircraft, the change of the first image acquisition direction is decoupled from the flight speed direction of the aircraft, and the first image is used to assist in obstacle avoidance.

[0013] In a fifth aspect, an embodiment of the present application provides a method for processing auxiliary images, comprising: obtaining image information captured by multiple first image acquisition devices of an aircraft, wherein the multiple first image acquisition devices are used to sense obstacles around the aircraft; dividing the image information captured by the first image acquisition device used for obstacle avoidance in the current flight speed direction of the aircraft into two transmission paths, wherein the image information transmitted in one path is used to sense obstacles around the aircraft, so that the aircraft can autonomously avoid obstacles based on the image information, and at least part of the image of the image information transmitted in the other path is used as a first image, wherein the first image is used for viewing by a user, and the second image acquisition direction corresponding to the first image changes with the change of the flight speed direction; outputting the first image; wherein, when the change of the flight speed direction is always within the field of view of the same first image acquisition device, the first image always comes from the same first image acquisition device.

[0014] In the sixth aspect, an embodiment of the present application provides an auxiliary image processing device, comprising a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and, when executing the computer program, implement the method steps described in the first to fifth aspects.

[0015] In the seventh aspect, an embodiment of the present application provides an aircraft, comprising a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and, when executing the computer program, implement the method steps described in aspects 1 to 5.

[0016] In an eighth aspect, an embodiment of the present application provides a control terminal comprising a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and, when executing the computer program, implement the method steps described in aspects 1 to 5.

[0017] In the ninth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the method steps described in the first to fifth aspects.

[0018] The embodiment of the present application can effectively help users view real-time environmental images in the current flight speed direction by outputting images associated with the current flight speed direction, so that users can control the aircraft according to the environmental images, avoiding safety issues caused by collisions due to sudden changes in the flight speed direction, and improving the safety of the aircraft during flight.

[0019] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] FIG1 is a schematic diagram of an application scenario of a control system provided in an embodiment of the present application;

[0022] FIG2 is a flowchart of one possible step of a method for processing auxiliary images provided in an embodiment of the present application;

[0023] FIG3 is a schematic diagram of an aircraft provided in an embodiment of the present application;

[0024] FIG4 is a schematic diagram of another aircraft scene provided by an embodiment of the present application;

[0025] FIG5 is a schematic diagram of another aircraft scene provided in an embodiment of the present application;

[0026] FIG6 is a schematic diagram of another aircraft scene provided in an embodiment of the present application;

[0027] FIG7 is a schematic diagram of another aircraft scene provided in an embodiment of the present application;

[0028] FIG8 is a schematic diagram of an interaction provided by an embodiment of the present application;

[0029] FIG9 is a schematic diagram of an interaction provided by an embodiment of the present application;

[0030] FIG10 is a flowchart of one possible step of a method for processing auxiliary images provided in an embodiment of the present application;

[0031] FIG11 is a flowchart of one possible step of a method for processing auxiliary images provided in an embodiment of the present application;

[0032] FIG12 is a flowchart of one possible step of a method for processing auxiliary images provided in an embodiment of the present application;

[0033] FIG13 is a flowchart of one possible step of a method for processing auxiliary images provided in an embodiment of the present application;

[0034] FIG14 is a schematic structural diagram of an auxiliary image processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0036] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined, or partially merged, so the actual execution order may vary depending on the actual situation.

[0037] With the development of technologies such as transportation vehicles and robots, users are increasingly concerned about their operational safety. Vehicles, constrained by their steering wheels and drive mechanisms, have relatively fixed steering angles and movement paths. Therefore, they can observe their surroundings through rearview mirrors facing in specific directions. Vehicles can also display images from cameras on both sides of the vehicle, allowing users to observe blind spots during steering.

[0038] However, robots and multi-rotor aircraft with universal wheels offer flexible and adaptable motion paths and steering angles, enabling parallel and diagonal movement, which is unattainable for vehicles. However, viewing the surrounding environment through a fixed-viewing angle can lead to a sudden change in direction, preventing the user from observing the surrounding environment, and potentially causing safety accidents.

[0039] Based on this, the present application provides an auxiliary image processing method, which aims to solve the technical problem that the environment in the corresponding direction cannot be observed due to changes in the moving direction during the driving of the movable platform, resulting in low driving safety.

[0040] The above method can be executed by a movable platform or control terminal in the control system. The movable platform can be a manned platform or an unmanned platform. The movable platform can be a land movable platform, such as a land robot or a car; the movable platform can also be an above-water or underwater movable platform; the movable platform can also be an aerial movable platform, such as an aircraft, including a multi-rotor aircraft, a fixed-wing aircraft, a helicopter, etc. The movable platform can also be an airborne and amphibious movable platform, such as a flying car. The following uses the movable platform as an aircraft as an example to explain the implementation method of the present application.

[0041] Please refer to Figure 1, which is a structural diagram of a control system provided by an embodiment of the present application, wherein the control system may include an aircraft and a control terminal.

[0042] As shown in FIG1 , a control system may include an aircraft 100 and a control terminal 200, and the aircraft 100 may be communicatively connected to the control terminal 200. The control terminal 200 may be used to control the aircraft 100. The control terminal 200 may include at least one of a remote control, a smartphone, and a tablet computer, or may include at least one of a remote control, a smartphone, and a wearable device, wherein the wearable device includes a head-mounted display device, and the head-mounted display device may include a virtual reality (VR) display device or a first-person view (FPV) display device. In some embodiments, the control terminal 200 may also include a terminal device for controlling the aircraft, such as an operating device on a manned aircraft.

[0043] In some embodiments, aircraft 100 includes a fuselage 110, a power system 120, an imaging device 130, and a control terminal (not shown in FIG1 ). The fuselage 110 may include a nose. In some embodiments, aircraft 100 further includes an arm connected to the fuselage 110 and used to mount the power system. In some embodiments, the power system 120 may be mounted directly on the fuselage 110.

[0044] The power system 120 is used to provide flight power for the aircraft. The power system 120 may include a motor and a propeller mounted on the motor and driven by the motor. The power system 120 can drive the fuselage 110 of the aircraft 100 to rotate around one or more rotation axes. For example, the above-mentioned rotation axes may include a roll axis, a yaw axis, and a pitch axis. When the power system 120 drives the fuselage 110 to rotate around the yaw axis, the yaw direction of the nose of the fuselage will change, that is, the yaw rotation of the fuselage 110 can be controlled by controlling the power system 120. It should be understood that the motor can be a DC motor or an AC motor. In addition, the motor can be a brushless motor or a brushed motor.

[0045] The camera 130 is directly carried on the fuselage 110 or carried through a gimbal, and is used to capture images, which may be pictures and / or videos. In some embodiments, as shown in FIG1 , the aircraft may include a gimbal 140, on which the camera 130 is mounted, and the gimbal 140 is connected to the fuselage 110. In some embodiments, the gimbal 140 is capable of controlling the yaw rotation of the camera 130 to adjust the yaw orientation of the camera 130. Specifically, the gimbal 140 may include a yaw motor 141, which is used to control the yaw rotation of the camera 130. In some embodiments, the gimbal 140 is capable of controlling the pitch rotation of the camera 130 to adjust the pitch orientation of the camera 130. Specifically, the gimbal 140 may include a pitch motor, which is used to control the pitch rotation of the camera 130. In some embodiments, the gimbal 140 can control the roll rotation of the camera 130 to adjust the roll direction of the camera 130. Specifically, the gimbal 140 may include a roll motor and a pitch motor for controlling the roll rotation of the camera 130.

[0046] In the yaw direction, the yaw rotation of the camera 130 and the yaw rotation of the body 110 may be associated. Further, the camera 130 may yaw and rotate following the yaw rotation of the body 110 , or the body 110 may yaw and rotate following the yaw rotation of the camera 130 .

[0047] The control terminal may include an input device, wherein the input device may detect control operations performed by a user of the control terminal. The control terminal may generate control instructions for the aircraft based on the user's control operations detected by the input device. For example, the control terminal may generate a yaw control instruction based on a yaw control operation performed by the user of the control terminal detected by the input device, and may transmit the yaw control instruction to the aircraft. Alternatively, the control terminal may generate a pitch control instruction based on a gimbal pitch control operation performed by the user of the control terminal detected by the input device, and may transmit the gimbal pitch control instruction to the aircraft.

[0048] Referring to FIG. 1 , in some embodiments, the control terminal 200 includes a remote controller, which is provided with an input device 210 and a communication device 220. The communication device 220 is a wireless communication device that may include at least one of a high-frequency radio transceiver, a Wi-Fi module, and a Bluetooth module. The input device 210 is used to generate corresponding control instructions in response to user manipulation, so that the remote controller can control the aircraft to adjust its flight attitude and / or flight speed through the control instructions. The input device 210 includes at least one of a button, a joystick, a dial, and a touch screen. For example, the input device 210 is a joystick mounted on the main body of the remote controller, and the remote controller senses the user's manipulation of the joystick and generates corresponding control instructions.

[0049] In some embodiments, the control terminal 200 can receive images transmitted by the aircraft 100 and display them on the display device 230. The display device 230 can be integrated into the control terminal 200, or the display device 230 can be set separately from the control terminal 200 and communicated with the control terminal 200. The communication connection method can be a wired communication connection method or a wireless communication connection method. For example, the wireless communication connection method can be a WiFi connection, a Bluetooth connection, or a high-frequency wireless signal connection.

[0050] In some embodiments, the control terminal 200 includes a remote controller, which is provided with an input device 210 and a communication device 220. The communication device 220 is a wireless communication device, which may include at least one of a high-frequency radio transceiver, a WIFI module, and a Bluetooth module. The input device 210 is used to generate corresponding control instructions in response to user manipulation, so that the remote controller can control the aircraft to adjust its flight attitude and / or flight speed through the control instructions. The input device 210 includes at least one of a button, a joystick, a dial, and a touch screen. For example, the input device 210 is a joystick, which is mounted on the body of the remote controller. The user can generate control instructions by using buttons, a joystick, or a dial, or by inputting operations on the touch screen, which is not limited here.

[0051] The specific implementation of the present application will be described in detail below with reference to the accompanying drawings. It should be understood that this exemplary description does not constitute a limitation to the present application.

[0052] Referring to Figure 2, Figure 2 is a schematic diagram of a possible step flow diagram of an embodiment of the present application. As shown in the figure, the present application provides a method for processing auxiliary images, including:

[0053] Step 101: Obtain the current flight speed and direction of the aircraft.

[0054] Step 102: In response to a situation where the current flight speed direction of the aircraft is inconsistent with the first image acquisition direction of the aircraft, image information of a first perspective acquired by the image acquisition device of the aircraft in a second image acquisition direction is acquired, wherein the second image acquisition direction is associated with the current flight speed direction of the aircraft and is different from the first image acquisition direction.

[0055] Step 103: Determine image information of a second perspective from the image information of the first perspective based on the current flight speed direction of the aircraft.

[0056] Step 104: output image information of the second perspective, wherein the second perspective is included in the first perspective, and the image information of the second perspective is used to obtain the current environmental conditions of the flight speed and direction of the aircraft.

[0057] It should be noted that the above steps 101-103 can be executed by a control terminal wirelessly connected to the aircraft, or by the aircraft's own control device. Some steps can also be executed by the aircraft and the remaining steps can be executed by the control terminal, which is not limited here.

[0058] In step 101 , optionally, obtaining the flight speed direction of the aircraft may be performed in a variety of ways.

[0059] For example, when the above step 301 is executed by an aircraft, in some embodiments, the current movement speed direction of the aircraft can be obtained by obtaining a geographic location signal in real time, such as a GPS signal; in some embodiments, for a multi-rotor aircraft, the attitude information in the IMU can also be obtained to calculate the speed direction corresponding to the current attitude; in some embodiments, the current speed direction can also be determined by the acceleration sensor in the IMU or by calculating the motor output, which is not limited here.

[0060] In some scenarios, the first image acquisition direction may be the shooting direction of a camera mounted on an aircraft, that is, the aircraft may have a shooting function, and the first image acquisition direction is the image acquisition direction of the aircraft's main camera. For example, when the aircraft is an unmanned platform, users can often observe the aircraft's surroundings through the images captured by the camera to assist in flight control. In some scenarios, the first image acquisition direction may change as the posture / orientation of the camera changes. However, because the camera needs to take into account the shooting effect, the user often cannot fully observe the aircraft's surroundings, which leads to safety issues.

[0061] In some scenarios, the first image acquisition direction may be the perception direction of an image acquisition device used by the aircraft for environmental perception. For example, the first image acquisition direction may be the perception direction of a monocular image acquisition device. The image acquisition device may include an image sensor and a lens. The lens and image sensor can be used to determine the viewing angle (maximum FOV) and current viewing angle of the image acquisition device, and the image acquisition direction can be determined based on the current viewing angle. The image acquisition device may be a wide-angle camera, a fisheye camera, or other similar camera, which are not listed here.

[0062] The processor can obtain image information different from the first image acquisition direction when the flight speed direction of the aircraft is inconsistent with the first image acquisition direction, so that the user can observe a more comprehensive environment around the aircraft. The first perspective can be the viewing angle range (maximum FOV) of the image acquisition device, or it can be a perspective determined by an algorithm. The image information of the first perspective can be acquired by a single image acquisition device, or it can be acquired by multiple image acquisition devices based on image stitching, image synthesis, etc. Referring to Figure 3, the first perspective in Figure 3 can be either F2 or F3, or it can be the perspective of F1 stitched together by F2 and F3. That is, the first perspective can be the perspective of a single image acquisition device, or it can be the perspective stitched together by multiple image acquisition devices, which is not limited here.

[0063] It should be understood that the first viewing angle corresponds to the second image acquisition direction, and the second image acquisition direction is different from the first image acquisition direction. Specifically, the central axis of the first viewing angle may substantially coincide with the second image acquisition direction, or the second image acquisition direction may be located at the center of the first viewing angle.

[0064] In step 103, the first perspective may be the optical field of view of the image capture device, which means that the first perspective may be large and inconvenient for display or user viewing. However, for an aircraft, observation of the environment in the direction of flight speed is very important during travel. Therefore, the image information from the first perspective can be processed to obtain image information from a second perspective corresponding to the direction of flight speed. The second perspective can be a partial perspective within the first perspective.

[0065] The flight speed direction of the aircraft may change within the first viewing angle and may always be within the second viewing angle during the change. In some embodiments, the central axis of the second viewing angle may coincide with the flight speed direction. In some embodiments, the flight speed direction may be located in the middle of the second viewing angle. The second viewing angle may be smaller than the first viewing angle and be within the range of the first viewing angle. Specifically, the second viewing angle may be a partial viewing angle in the first viewing angle. For example, when the first viewing angle is the maximum field of view of an image acquisition device, the second viewing angle may be a smaller field of view obtained by zooming or cropping the image of the image acquisition device; when the first viewing angle is the viewing angle obtained by combining multiple image acquisition devices, the second viewing angle may be the field of view corresponding to one of the sensors. These are not listed here one by one.

[0066] By outputting the image information of the second perspective in step 104, the user can clearly view and respond to the environmental image of the aircraft along the flight speed direction. This can effectively avoid the safety problem of collision along the flight speed direction caused by the inability of the image acquisition device with a fixed perspective to fully observe the environmental image in the flight speed direction of the aircraft when the flight speed direction of the aircraft frequently changes.

[0067] It should be understood that the flight velocity direction indicates the direction of change in the aircraft's body position. For four-wheel drive vehicles and fixed-wing aircraft, the flight velocity direction can be associated with the vehicle's nose orientation. For multirotor aircraft, the flight velocity direction can be decoupled from the nose orientation, allowing for more flexible changes in motion direction. For example, drones can fly inverted, sideways, or at an angle.

[0068] In some embodiments, the flight speed direction may include a horizontal component direction and a vertical component direction. Specifically, the horizontal component direction may include any one of front, back, left, and right. The vertical component direction may include up or down.

[0069] For an aerial vehicle, the direction of its motion speed may include both the horizontal and vertical directions along the fuselage, so it is necessary to consider both the horizontal and vertical environmental information. The above step 102 may specifically include:

[0070] Based on the horizontal component direction and the vertical component direction of the flight speed direction of the aircraft, the image information of the second perspective is determined from the image information of the first perspective.

[0071] Referring to FIG4 , the aircraft in FIG4 may be provided with image acquisition devices on the upper and lower sides. The first viewing angle may be the viewing angle A in FIG4 , and the flight speed direction V1 of the aircraft includes an upward vertical component V 1U and the rightward horizontal component V 1R The second perspective can be the B perspective in Figure 4. The B perspective also changes with the direction of the flight speed. Furthermore, the central axis of the B perspective can be the direction of the aircraft's flight speed. This allows the image information from the B perspective to simultaneously understand the environment in front of and above the aircraft, further enhancing aircraft safety.

[0072] Optionally, the aircraft can be equipped with multiple image sensors positioned in different locations and facing different directions to achieve multi-directional environmental perception. For example, referring to Figure 5 , four image capture devices can be installed at the four corners of the aircraft. Each image capture device can have a 180° viewing angle (F1-F4) in the circumferential direction. These four image capture devices can output omnidirectional images of the environment in the horizontal direction. Image capture devices can also be installed on the upper and lower sides of the aircraft to respectively sense the environment in the vertical direction.

[0073] Optionally, the image information of the second perspective may be acquired by one or more image acquisition devices.

[0074] In some embodiments, the flight speed direction of the aircraft can be in the horizontal direction and the vertical direction. In order to take into account the horizontal and vertical environments of the aircraft, the image information of the second perspective may include image information obtained by at least two image acquisition devices, and the acquisition directions of the at least two image acquisition devices correspond to the horizontal component direction and the vertical component direction, respectively.

[0075] Of course, in some embodiments, in order to take into account the horizontal and vertical environments of the aircraft, images in both horizontal and vertical directions can be acquired by setting an image acquisition device tilted at the corner of the aircraft. The specific setting can be made according to actual conditions.

[0076] Since the flight speed and direction of the aircraft can change in real time, optionally, the second perspective can also change with the flight speed and direction. In some embodiments, the method may further include:

[0077] Change the second perspective based on the change in flight speed and direction.

[0078] Furthermore, based on the change in the flight speed direction, changing the second viewing angle may specifically include at least one of the following:

[0079] Based on the change in the flight speed direction, changing the image acquisition device corresponding to the second viewing angle;

[0080] Based on the change in the flight speed direction, changing the image acquisition direction of the image acquisition device corresponding to the second viewing angle;

[0081] Change the size of the second perspective based on the change in flight speed and direction.

[0082] Continuing with FIG. 4 , the flight speed direction of the aircraft in FIG. 4 changes from V1 to V2. The image acquisition device corresponding to the second perspective can be transformed from an upper image acquisition device to a lower image acquisition device. Alternatively, in some embodiments, the image acquisition device corresponding to the second perspective can adjust the image acquisition direction by changing its own posture, receiving / reflecting light path, changing the control method of the sensor, etc., so that the corresponding second perspective changes with the flight speed direction. Alternatively, the second perspective can change its own viewing angle according to the change in the flight speed direction, for example, by increasing the viewing angle to better cover the environment image along the flight speed direction.

[0083] For example, in some embodiments, the movable platform may control the image acquisition device to change the viewing angle corresponding to the first direction to the viewing angle corresponding to the second direction when the flight speed direction changes from the first direction to the second direction.

[0084] Continuing with FIG. 3 , in one specific embodiment, the vertical components of the first direction V1 and the second direction V2 in FIG. 3 are different. When the flight velocity direction changes from the first direction to the second direction, the second perspective corresponding to the aircraft moving in the second direction may change at least along the pitch axis or the height axis of the aircraft body compared to when the aircraft moves in the first direction.

[0085] In other optional embodiments, when one of the first direction and the second direction includes only a horizontal component direction and the other includes a horizontal component direction and a vertical component direction, the second viewing angle corresponding to the movement of the aircraft along the second direction may also change at least along the pitch axis direction or the height direction of the aircraft's fuselage compared to when the aircraft moves along the first direction.

[0086] In this way, when the aircraft switches from moving in the horizontal direction to moving in at least the vertical direction, the second viewing angle can change along the pitch axis and / or along the height direction of the aircraft to raise or lower the viewing angle to ensure the capture of images in the vertical direction.

[0087] In some embodiments, during the change of the second perspective, an image between the two perspectives can be further displayed to make the image smoother during the change and easier for the user to observe. Controlling the image acquisition device to change from the perspective corresponding to the first direction to the perspective corresponding to the second direction can specifically include:

[0088] Acquire image information of a viewing angle between the viewing angle corresponding to the first direction and the viewing angle corresponding to the second direction;

[0089] When the image acquisition device changes from a viewing angle corresponding to a first direction to a viewing angle corresponding to a second direction, image information of a viewing angle between the viewing angle corresponding to the first direction and the viewing angle corresponding to the second direction is output.

[0090] Optionally, the triggering condition for changing the second perspective can be determined based on changes in flight speed to adapt to actual scenarios. In some embodiments, an aircraft may frequently make small changes in direction. To avoid rapid image changes caused by frequent perspective switching, the second perspective is changed based on changes in flight speed direction, including: changing the second perspective when the flight speed direction changes and the speed magnitude exceeds a preset threshold.

[0091] In some embodiments, in order to avoid frequent switching of perspectives causing the image to change too quickly based on the change in the flight speed direction, the second perspective is changed based on the change in the flight speed direction. It may also include: changing the second perspective when the flight speed direction changes and the time length for maintaining the changed flight speed direction is greater than a preset threshold.

[0092] In some embodiments, the aircraft may move along a preset trajectory. To facilitate the user's understanding of the environment corresponding to the aircraft's upcoming change in flight speed and direction, the second perspective may be changed based on the change in flight speed and direction, including:

[0093] During the movement of the aircraft along a preset trajectory, the flight speed direction changes based on the preset trajectory prediction; before the flight speed direction changes, the second perspective changes based on the predicted flight speed direction. The preset trajectory can be a trajectory pre-set by user input, or a path file sent by another device. The preset trajectory can include the position information of multiple trajectory points, so that the aircraft's flight speed direction can be determined based on the position information of two adjacent trajectory points, and the change in the aircraft's flight speed direction after reaching the next waypoint can be predicted. Changing the second perspective can be performed upon reaching a waypoint or before reaching a waypoint, and is not limited here.

[0094] In this way, while the aircraft moves along the preset trajectory, it can show the user in advance the image corresponding to the direction of the flight speed that will be changed based on the predicted flight speed direction, so that the user can monitor the environmental information in the direction the aircraft will go and make timely control responses.

[0095] Since a single image sensor simultaneously captures images of the aircraft in both the horizontal and vertical directions, the images may be blocked by the aircraft body, thus affecting the image quality. In some embodiments, the second viewing angle is changed based on the change in the flight speed direction, including:

[0096] When the motion speed changes from containing only horizontal components or vertical components to containing both horizontal and vertical components, the aircraft is controlled to change from obtaining image information of the second perspective from one sensor to obtaining image information of the second perspective from at least two sensors.

[0097] This allows the aircraft to capture horizontal or vertical image information using a single image sensor when moving horizontally or vertically, effectively reducing resource usage. When the aircraft has both horizontal and vertical components, it can capture image information using multiple image sensors to avoid obstruction by the aircraft body and provide a more comprehensive image of the environment.

[0098] Furthermore, at least two sensors may be used to acquire image information along the horizontal direction and image information along the vertical direction, respectively, to achieve a greater degree of environmental image coverage.

[0099] As can be seen from the above, the image acquisition device can adjust the second perspective by changing its posture or other methods. The second perspective can also be changed by changing the image acquisition device corresponding to the second perspective. In some embodiments, changing the second perspective based on changes in the direction of flight speed includes:

[0100] When the change in the flight speed direction is less than a preset threshold, the second viewing angle is changed by changing the imaging range of the image acquisition device that acquires the image information of the second viewing angle and / or changing the posture of the image acquisition device that acquires the image information of the second viewing angle;

[0101] When the change in the flight speed direction is greater than or equal to a preset threshold, the image acquisition device for acquiring image information of the second viewing angle is changed. The preset threshold may be an angle threshold.

[0102] In this way, when the aircraft's flight speed and direction change slightly, the second perspective can be transformed by adjusting the image acquisition direction, which helps improve efficiency and reduce resource usage. In the event of a significant change in the aircraft's flight speed and direction, by changing the image acquisition device corresponding to the second perspective, the structural and hardware limitations of adjusting the image acquisition direction using only a single image acquisition device can be avoided.

[0103] In some scenarios, users need to see a wider range of images, but the imaging range of the images captured by the perspective of a single image capture device may be smaller than the range required by the user. Because the image information from the second perspective may include image information captured by one or more image capture devices, in some embodiments, determining the image information from the first perspective based on the flight speed and direction of the aircraft includes:

[0104] When the second viewing angle is greater than the viewing angle of a corresponding image acquisition device in the aircraft, image information of the second viewing angle is determined based on image information acquired by at least two image acquisition devices in the aircraft.

[0105] Further, referring to Figure 5, if the aircraft obtains image information of the front through two image acquisition devices, the overlapping angle of the viewing angle between the two image acquisition devices is 90°, and if the image viewing angle required by the user is 110°, then in order to ensure that a 110° image can be obtained when the aircraft moves forward at any angle, the image information obtained by the two image acquisition devices can be used to determine the image information of the second viewing angle.

[0106] In some embodiments, to reduce the resource consumption associated with simultaneously acquiring image information from multiple image sensors, one or more image acquisition devices of an aircraft can be determined based on the direction of flight speed. In some embodiments, the angle between the optical axis of one or more image acquisition devices and the direction of flight speed is smaller than the angle between the optical axes of the other image acquisition devices of the aircraft. For example, referring to Figure 6 , if the aircraft is moving in a forward and downward direction, only image information from the forward image acquisition device A and the downward image acquisition device B can be acquired, and the image information for the second perspective can be determined based on this image information. This effectively reduces the resource consumption of outputting the second perspective image, improving efficiency.

[0107] Similar to the second perspective, in some embodiments, the image information of the first perspective can also be synthesized from image information of multiple image acquisition devices with different perspectives. To avoid repetition, it will not be described here.

[0108] Optionally, the method for determining the second-perspective image information from the first-perspective image information can be configured based on actual needs. In some embodiments, the second-perspective image information is obtained by cropping the first-perspective image information. In some embodiments, the second-perspective image information is obtained by partially rendering the first-perspective image information, which are not listed here one by one.

[0109] As described above, the image acquisition device can output the collected image information as an image to assist user control. In some embodiments, the image acquisition device can also be used to output image information for autonomous obstacle avoidance. Specifically, the above image information can be combined with obstacle recognition algorithms, depth information, etc. to achieve the identification of obstacles and obstacle distances, so as to automatically control the aircraft to perform autonomous obstacle avoidance operations. In this way, the above image information can be used by the user to view and realize the identification of obstacles and obstacle distances, and autonomous obstacle avoidance, thereby improving the utilization rate of image information and reducing resource usage.

[0110] In some embodiments, to ensure that the image viewed by the user does not experience significant jitter due to the movement of the aircraft, which could affect viewing, the second-perspective image information is derived from the image information used for obstacle avoidance after stabilization. Specifically, the second-perspective image information can be stabilized based on the aircraft's attitude information. In some embodiments, before outputting the second-perspective image information, the method may further include: obtaining the aircraft's attitude information; and stabilizing the second-perspective image information based on the attitude information.

[0111] As mentioned above, in addition to acquiring surrounding environment images through the image acquisition device to perceive the surrounding environment, the aircraft can also use the image acquisition device to take pictures.

[0112] In some embodiments, the method may further include: acquiring image information from a third perspective using an image acquisition device of the aircraft; and outputting the image information from the third perspective, wherein the third perspective may be determined based on a shooting direction. The shooting direction may be the first image acquisition direction. Continuing with FIG. 6 , the perspective of the image acquisition device C in FIG. 6 may be the third perspective.

[0113] Some image acquisition devices can have a viewing angle of up to 360°, or a panoramic range. Referring to FIG4 , the aircraft shown in FIG4 can capture panoramic images using upper and lower image acquisition devices. Therefore, the aforementioned first viewing angle can be a larger viewing angle, such as the upper hemisphere or lower hemisphere viewing angle shown in FIG4 . In some embodiments, to improve the utilization of image information, image information from a third viewing angle can also be obtained from the image information from the first viewing angle.

[0114] In some embodiments, the image information of the third perspective can also be obtained based on the image acquisition device corresponding to the shooting device of the aircraft to obtain an image with higher shooting quality.

[0115] In some embodiments, the image information of the first perspective may have color information, and the image generated based on the image information of the second perspective may be a color image to enhance the user's viewing experience; in some embodiments, the image information of the first perspective may also have only grayscale information, and the image generated based on the image information of the second perspective may be a grayscale image to reduce costs and resource usage, which is not limited here.

[0116] Optionally, the above-mentioned method of outputting the image information of the second viewing angle can be set according to actual needs.

[0117] In some embodiments, the image information of the second viewing angle may be sent to a monitoring / control terminal so that the user can view it in real time.

[0118] As for real-time image transmission, if it is necessary to display images from multiple perspectives on the aircraft at the same time, this places high demands on the image transmission bandwidth and the processing capabilities of the control terminal.

[0119] To reduce image transmission requirements, in some embodiments, the third-perspective image information and the second-perspective image information can be transmitted via a single image transmission channel. Specifically, the third-perspective image information and the second-perspective image information can be spliced ​​together, and the spliced ​​image can be sent to the control terminal via a single image transmission channel. After decoding by the control terminal, the spliced ​​image is cropped to display the second-perspective and third-perspective images, respectively. This reduces the image transmission bandwidth requirements and the control terminal hardware requirements, thereby reducing costs.

[0120] As mentioned above, the aircraft may be an airplane. For an aircraft, the direction of its flight velocity may include a vertical component, and the aircraft may also have several flight modes, such as landing and circling. Optionally, the first image may include image information in addition to image information along the direction of the flight velocity, as well as image information in other directions to accommodate the aircraft's flight mode. Specifically, the first image may be synthesized or spliced ​​from image information in multiple directions.

[0121] In some embodiments, when the aircraft is in the process of returning to the destination, the image information of the second perspective includes at least one of the following:

[0122] Image information below the aircraft; image information along the direction toward the home point. Specifically, the second-perspective image information can be synthesized / joined by at least the above two image information. Thus, during the aircraft's return, the second-perspective image information can simultaneously include image information below the aircraft and image information in the direction of the aircraft. This allows the user to simultaneously view the aircraft's heading, the environment below, and the home point from a single image, allowing the user to better control the aircraft for landing.

[0123] In some embodiments, while the aircraft is landing, the second perspective image information may include image information from at least one of the aircraft's surrounding directions and / or image information from below the aircraft. This allows the user to simultaneously view both the environment below the aircraft and the surrounding environment from a single image. The user can adjust the landing position based on the downward view from the aircraft and avoid collisions when adjusting the landing position based on the surrounding view, thereby better controlling the aircraft for landing.

[0124] In some embodiments, while an aircraft is orbiting a target object, the image information from the second perspective also includes at least one of the following: image information perpendicular to the aircraft's flight velocity; or image information at a fixed angle to the flight velocity. Referring to Figure 7 , the aircraft is orbiting the target object in a clockwise direction. In this case, the aircraft's flight velocity is along a tangent to the circumference, and obstacles exist both inside and outside the aircraft. By displaying image information perpendicular to the aircraft's flight velocity (the X or Y direction in the figure) and / or image information at a fixed angle to the flight velocity, safety issues caused by the aircraft's incomplete observation of the environment inside / outside the orbiting path can be avoided during the orbit. In some embodiments, while an aircraft is tracking a target object, the image information from the second perspective includes image information along the aircraft's flight velocity and / or image information along a direction toward the target object. The aircraft can track the target object using image information captured by an image acquisition device or using a positioning device carried by the target object. When the aircraft is tracking the target object using a positioning device carried by the target, the user can use the image acquisition device to simultaneously observe the environment, the target object, and the flight speed and direction, allowing them to adjust the tracking path and composition, eliminating the problem of unclear relative position between the target object and the aircraft. This can also effectively improve safety in scenarios where positioning device errors are large or fail, while also enhancing the user experience.

[0125] Optionally, the image information of the second perspective may be outputted by sending the image information of the second perspective to an external device; or may be outputted by displaying an image corresponding to the image information of the second perspective.

[0126] In some embodiments, outputting the image information of the second perspective includes: displaying a first image corresponding to the image information of the second perspective and a second image corresponding to the image information of the third perspective.

[0127] Furthermore, in some embodiments, when a first image and a second image are displayed on the same interface, the first image may occupy a portion of the second image, or the second image may occupy a portion of the first image. Referring to FIG8 , the small image in the lower left corner of FIG8 may be the first image, and the large image may be the second image. Thus, by overlapping the first and second images, the image's occupancy on the display interface can be reduced, and the image can be made into a regular shape, making it easier for the control terminal to display it on the display interface.

[0128] Optionally, the display ratio, display position, etc. of the first image and the second image can be switched based on user input. In some embodiments, the method may further include:

[0129] receiving a first input from a user;

[0130] In response to the first input, the first image and the second image are switched and displayed.

[0131] We can further refer to Figure 8, where the first input can be a touch input, such as a single-click, double-click, long press or short press on the first image, the second image or the switching control; the first input can also be a non-touch input, such as voice input, gesture input, etc., which are not listed one by one here.

[0132] Furthermore, switching the display may be switching the display ratio, switching the display interface, or switching the display size, etc. In some embodiments, switching the display of the first image and the second image includes: switching the display ratio of the first image and the second image in the same interface; or switching the display interface to display the first image or the second image.

[0133] Since the user can also view the surrounding environment by viewing the captured image, in some embodiments, the above method may further include:

[0134] The display modes of the first image and the second image are changed based on the shooting direction and the flying speed direction of the aircraft.

[0135] In some embodiments, when the shooting direction is consistent with the flight speed direction, the second image may be displayed first. In some embodiments, when the shooting direction is inconsistent with the flight speed direction, the first image may be displayed first.

[0136] Among them, the priority display can be to enlarge the display size of the image or only display the image. In this way, when the shooting direction is roughly consistent with the flight speed direction, the user can give priority to viewing the shooting effect of the shooting picture while taking into account the observation of the environment around the aircraft; when the shooting direction and the movement speed are inconsistent, the user can give priority to observing the environment around the aircraft to ensure safety.

[0137] In some scenarios, the user may need to observe an image with a fixed viewing angle. In some embodiments, the method further includes:

[0138] receiving a second input of the user regarding the first image;

[0139] In response to the second input, the viewing angle of the first image is controlled to remain unchanged.

[0140] The second input is similar to the first input, but can be different inputs. For example, with reference to FIG9 , the user can lock the current perspective of the first image by long pressing the first image, so that the first image always maintains the locked perspective before being unlocked. For example, when the aircraft is flying along the left front as a whole, there may be some moments when it flies to the right front. At these moments, the perspective of the first image can be locked along the left front. In this way, the user can observe the picture with a fixed perspective through input, so that the user can observe the fixed perspective direction of the aircraft in some scenes, and avoid the change of image perspective caused by the change of aircraft speed direction.

[0141] Of course, in other optional embodiments, a new display screen with a fixed viewing angle may be added so that the user can continue to observe the screen with a fixed viewing angle, which is not limited here.

[0142] In some scenarios, the user may need to briefly observe images in other directions of the aircraft. For example, before executing a turn, the user may need to first observe the environment in the direction of the turn. By changing the perspective of the first image, the user can observe the environment in other directions of the aircraft.

[0143] Continuing with FIG. 8 , in some embodiments, the method further includes:

[0144] receiving a third input from the user regarding the first image;

[0145] In response to a trigger of a third input, changing the viewing angle of the first image;

[0146] In response to the end of the third input, the viewing angle of the first image is restored.

[0147] In Figure 8 , the user can touch the directional control on the first image to change the direction of the first image's perspective. Furthermore, the extent of the change in perspective can be determined based on the duration of the touch. For example, in some embodiments, the longer the directional control is touched, the greater the angle of the first image's perspective shifts from the original perspective. When the user stops touching the directional control, the perspective of the first image is restored to the state before receiving the third input. This allows the user to observe the environment from other perspectives at any time using the third input, and can restore the perspective by stopping the third input, improving operational convenience for the user when viewing the aircraft's surroundings.

[0148] Of course, in some embodiments, when the user touches the direction control on the first image, the perspective of the first image can be changed to a fixed perspective corresponding to that direction. When the user long presses the direction control of the first image for a preset time, the perspective of the first image is locked to that perspective, so that the user can change and lock the perspective of the first image with one touch operation, thereby improving the convenience of operation.

[0149] Optionally, obstacle information can be displayed on the first image to facilitate user obstacle identification, enabling the user to more promptly execute obstacle avoidance maneuvers. In some embodiments, the method may further include displaying the first image based on obstacle information along the aircraft's flight velocity. The obstacle information may include obstacle outline information, obstacle distance information, obstacle type information, and other information. The obstacle information may be displayed using numerical annotation, color annotation, textual prompts, or a zoomed-in display, among other methods, without limitation.

[0150] In some embodiments, the method also includes: when the distance between the aircraft and an obstacle along the viewing direction of the first image is less than a preset threshold, displaying prompt information on the first image, so that the user can be aware of the obstacle in the viewing direction of the aircraft in the first image and respond in time to improve the safety of the aircraft operation.

[0151] In some embodiments, in order to more intuitively display obstacles in the environment for the user to view, the above method may further include:

[0152] Based on obstacle information along the aircraft's flight speed, a second-perspective image is displayed. This second-perspective image can be displayed in conjunction with depth information, specifically by color or annotation of obstacle information and distance, allowing users to perform obstacle avoidance control based on the obstacle information.

[0153] In some embodiments, the user can change the size or orientation of the second perspective by inputting. Referring to Figure 8 or Figure 9, the user can adjust the second perspective by pinching to zoom, and change the orientation of the second perspective by clicking a direction in the display control.

[0154] It should be understood that the above-described user interaction method embodiments can be executed by a control terminal wirelessly connected to the aircraft, or by the aircraft's own control device, without limitation. In the embodiments of the present application, the display format of the second-perspective image information can be changed through user input, allowing users to select the image display format based on their actual needs, further enhancing the user experience.

[0155] 10 , an embodiment of the present application further provides a method for processing an auxiliary image, including:

[0156] Step 201: Acquire composite image information of a first perspective captured by at least two image capture devices of an aircraft.

[0157] Step 202: Determine image information of a second perspective from the synthesized image information of the first perspective based on the current flight speed and direction of the aircraft, wherein the second perspective is included in the first perspective;

[0158] Step 203: Output image information from the second perspective, wherein the image information from the second perspective is used to obtain the environmental conditions of the current flight speed and direction of the aircraft.

[0159] Unlike step 102 described above, the image information from the first perspective in step 201 can be synthesized from image information captured by at least two image capture devices. The image information from the second perspective in step 202 can be partial image information from the synthesized image information. By synthesizing the image information captured by multiple image capture devices and then determining and outputting a partial image from the synthesized image based on the flight speed, this avoids the problem of a single image capture device being unable to output image information outside its field of view due to field limitations, resulting in a smaller user-viewable area. This also allows the image information from the second perspective to be determined from an image covering a larger area, further facilitating consistent tracking of the second perspective image information even when the aircraft's flight speed fluctuates significantly and frequently.

[0160] It should be understood that the specific implementation of the method embodiment in steps 201-203 can refer to the explanation of the method embodiment in steps 101-103, and will not be described again here to avoid repetition.

[0161] 11 , an embodiment of the present application further provides a method for processing an auxiliary image, including:

[0162] Step 301: Obtain the current flight speed and direction of the aircraft.

[0163] Step 302: When the first image acquisition direction of the aircraft is inconsistent with the current flight speed direction of the aircraft, output the first image of the second image acquisition direction, wherein the second image acquisition direction is different from the first image acquisition direction, and the second image acquisition direction changes with the change of the flight speed direction of the aircraft.

[0164] When the above step 301 is executed by the control terminal, obtaining the current flight speed direction of the aircraft may be receiving flight speed information sent by the aircraft.

[0165] Outputting the first image in the second image acquisition direction may include displaying the first image in the second image acquisition direction, or may include sending the first image to an external device, which is not limited herein.

[0166] Optionally, the first image may be acquired by an image acquisition device of the aircraft, and specifically may be determined by image information from the first perspective in the aforementioned embodiment, or may also be determined by image information from the second perspective in the aforementioned embodiment. The first image acquisition direction and the second image acquisition direction may refer to the explanations in the aforementioned embodiment, and to avoid repetition, they are not further described here.

[0167] In the embodiment of the present application, the flight speed direction of the aircraft is obtained through step 301, and when the first image acquisition direction of the aircraft is inconsistent with the flight speed direction of the aircraft, the first image of the second image acquisition direction acquired by the aircraft is output through step 302, so that the user can obtain environmental information corresponding to the direction of the aircraft through the first image, which is convenient for the user to perform corresponding control and improve the safety of the aircraft's operation.

[0168] In some embodiments, the second image acquisition direction may be determined based on the vertical flight speed direction and the horizontal flight speed direction of the aircraft.

[0169] In some embodiments, to facilitate user observation of the captured image, the method may further include outputting a second image, where the viewing angle of the second image corresponds to the shooting direction of the aircraft's main camera. Of course, in other optional embodiments, the second image may also be captured by an image capture device that does not change with flight speed, such as an image capture device with a fixed viewing angle.

[0170] In some embodiments, the flight speed direction can be used to indicate the direction of the change in the position of the aircraft's fuselage. In some embodiments, the flight speed direction can include a component along the horizontal direction of the aircraft and a component along the vertical direction of the aircraft. In some embodiments, the flight speed direction can include a horizontal component direction and a vertical component direction. In some embodiments, the first image includes image information along the horizontal component direction of the flight speed direction and image information along the vertical component direction of the flight speed direction. In some embodiments, the horizontal component direction can include at least one of front, back, left, and right. In some embodiments, the vertical component direction can include up and / or down.

[0171] In some embodiments, the aircraft includes a first image acquisition device and a second image acquisition device, the first image is acquired based on the first image acquisition device, the second image is acquired based on the second image acquisition device, and the perspective of the first image acquisition device and the perspective of the second image acquisition device are facing different directions.

[0172] In some embodiments, the first image acquisition device is used to perceive the environment surrounding the aircraft.

[0173] In some embodiments, the second image acquisition device is used to obtain image information from a camera mounted on the aircraft. In some embodiments, the first image is obtained based on image information having a first perspective, wherein the first image has a second perspective that is included in the first perspective.

[0174] In some embodiments, the first image is a partial image obtained by synthesizing image information from at least two image acquisition devices. In some embodiments, outputting the first image in the second image acquisition direction acquired by the aircraft includes: changing the imaging range of the first image based on a change in the direction of the aircraft's flight speed.

[0175] The imaging range of the first image can be determined based on the viewing angle and imaging distance of the image acquisition device that acquired the first image. Optionally, the method for changing the imaging range of the first image can be selected according to actual needs. In some embodiments, outputting the first image acquired by the aircraft in the second image acquisition direction can specifically include: changing the viewing angle of the image acquisition device corresponding to the first image based on changes in the aircraft's flight speed direction; changing the image acquisition device corresponding to the first image based on changes in the aircraft's flight speed direction; and changing the imaging range corresponding to the first image based on changes in the aircraft's flight speed direction.

[0176] In some embodiments, changing the imaging range of the first image based on a change in the flight speed direction of the aircraft includes: changing the imaging range of the first image when the flight speed direction changes and the movement speed is greater than a preset threshold.

[0177] In some embodiments, when the direction of the flight speed changes and the duration of the change is greater than a preset threshold, the imaging range of the first image is changed.

[0178] In some embodiments, during operation based on a preset trajectory, a speed change is predicted based on the preset trajectory, and the imaging range of the first image is changed before the speed change.

[0179] In some embodiments, when the motion speed changes from a first direction to a second direction, the viewing angle of the first image is controlled to change from a viewing angle corresponding to the first direction to a viewing angle corresponding to the second direction.

[0180] In some embodiments, when the flight speed direction changes from a first direction to a second direction, the imaging range of the first image when the aircraft moves along the second direction changes at least along the vertical direction of the aircraft compared to the imaging range when the aircraft moves along the first direction, wherein one of the first direction and the second direction includes only a horizontal component direction, and the other includes a horizontal component direction and a vertical component direction, or the vertical component directions of the first direction and the second direction are different.

[0181] In some embodiments, controlling the imaging range of the first image to change from the imaging range corresponding to the first direction to the imaging range corresponding to the second direction includes: displaying image information between the imaging range corresponding to the first direction and the imaging range corresponding to the second direction.

[0182] In some embodiments, outputting the first image in the second image acquisition direction acquired by the aircraft includes: displaying the first image and the second image on the same interface, and / or displaying the first image and the second image on different interfaces.

[0183] In some embodiments, when the first image and the second image are displayed on the same interface, the first image occupies a partial area of ​​the second image, or the second image occupies a partial area of ​​the first image.

[0184] In some embodiments, the method may further include:

[0185] receiving a first input from a user;

[0186] In response to the first input, the first image and the second image are switched and displayed.

[0187] In some embodiments, switching the display of the first image and the second image includes: switching the display ratio of the first image and the second image in the same interface; or switching the display interface to display the first image or the second image.

[0188] In some embodiments, the above method may further include:

[0189] The display modes of the first image and the second image are changed based on the shooting direction and the flying speed direction of the aircraft.

[0190] In some embodiments, when the shooting direction is consistent with the flight speed direction, the second image may be displayed first. In some embodiments, when the shooting direction is inconsistent with the flight speed direction, the first image may be displayed first.

[0191] In some embodiments, the method further comprises:

[0192] receiving a second input of the user regarding the first image;

[0193] In response to the second input, the viewing angle of the first image is controlled to remain unchanged.

[0194] In some embodiments, the method further comprises:

[0195] receiving a third input from the user regarding the first image;

[0196] In response to a trigger of a third input, changing the viewing angle of the first image;

[0197] In response to the end of the third input, the viewing angle of the first image is restored.

[0198] In some embodiments, when the user touches the direction control on the first image, the perspective of the first image can be changed to a fixed perspective corresponding to that direction. When the user long presses the direction control of the first image for a preset time, the perspective of the first image is locked to that perspective, so that the user can change and lock the perspective of the first image with one touch operation, thereby improving the convenience of operation.

[0199] In some embodiments, the above method may further include: displaying the first image based on obstacle information of the aircraft along the flight speed direction.

[0200] In some embodiments, the method also includes: when the distance between the aircraft and an obstacle along the viewing direction of the first image is less than a preset threshold, displaying prompt information on the first image, so that the user can be aware of the obstacle in the viewing direction of the aircraft in the first image and respond in time to improve the safety of the aircraft operation.

[0201] In some embodiments, the first image and the second image are received via a transmission channel. In some embodiments, the control terminal may further display a third image having a different acquisition direction from the first image and the second image, so that the user can view images in different directions in different scenarios, thereby improving the user experience. The method may further include:

[0202] A third image is displayed that is different from the flight speed direction and the shooting direction of the aircraft.

[0203] In some embodiments, the display viewing angle of the third image is determined based on the attitude of the control terminal of the aircraft; or,

[0204] The display viewing angle of the third image is determined based on the preset flight path direction of the aircraft.

[0205] Optionally, the first image may include not only image information along the flight speed direction but also image information in other directions to adapt to the flight mode of the aircraft. Specifically, the first image may be obtained by synthesizing or splicing image information from multiple directions.

[0206] In some embodiments, when the aircraft is in the process of returning to the home position, the first image may further include image information below the aircraft and / or image information in the direction of the return point.

[0207] In some embodiments, when the aircraft is in the process of landing, the first image may further include image information of at least one direction of the aircraft's surrounding viewing direction, and / or image information below the aircraft.

[0208] In some embodiments, when the aircraft is flying around the target object, the first image may further include image information perpendicular to the heading of the aircraft and / or image information at a fixed angle to the heading.

[0209] In some embodiments, when the aircraft is tracking a target object, the image information of the second perspective may further include image information along the direction of the aircraft's flight speed and / or image information in the direction toward the target object.

[0210] For the parts not mentioned in the embodiments of the present application, reference can be made to the relevant introduction of the embodiments described in the aforementioned steps 101-103. Without conflict, they can all be regarded as optional features of the embodiments of the present application and will not be described in detail here.

[0211] 12, the present application also provides a method for processing auxiliary images, including:

[0212] Step 401: Obtain the current flight speed and direction of the aircraft.

[0213] Step 402: Based on the current flight speed direction of the aircraft, output a first image in the second image acquisition direction and a second image in the first image acquisition direction, wherein the second image acquisition direction changes with the change of the flight speed direction of the aircraft, and the change of the first image acquisition direction is decoupled from the flight speed direction of the aircraft, and the first image is used to assist in obstacle avoidance.

[0214] Unlike steps 301-302 above, step 402 can simultaneously output the first and second images. The acquisition direction of the second image corresponding to the first image can change as the flight speed direction changes, while the change in the acquisition direction of the first image corresponding to the second image is decoupled from the flight speed direction. This allows the user to simultaneously view both the environmental image associated with the flight speed direction and the environmental image decoupled from the flight speed direction. This allows the user to simultaneously observe the environmental image associated with the flight speed direction in real time while also stably viewing the image decoupled from the flight speed direction changes, further enhancing the user experience.

[0215] The second image can be a live-view image captured by a camera mounted on the aircraft. The live-view image does not change with flight speed, helping the user better observe the captured image. In other optional embodiments, the second image can also be another image decoupled from flight speed, such as an image captured by a fixed-viewing angle image acquisition device. The specific implementation of steps 401-402 can be referenced to the explanation of steps 301-302 above and will not be repeated here to avoid repetition.

[0216] For the parts not mentioned in the embodiments of the present application, reference can be made to the relevant introduction of the embodiments described in the aforementioned steps 101-103. Without conflict, they can all be regarded as optional features of the embodiments of the present application and will not be described in detail here.

[0217] 13, the present application also provides a method for processing auxiliary images, including:

[0218] Step 501: Acquire image information collected by multiple first image acquisition devices of an aircraft, wherein the multiple first image acquisition devices are used to sense obstacles around the aircraft.

[0219] Step 502: The image information collected by the first image acquisition device for obstacle avoidance in the current flight speed direction of the aircraft is divided into two transmission paths. The image information transmitted in one path is used to sense obstacles around the aircraft so that the aircraft can autonomously avoid obstacles based on the image information. At least part of the image information transmitted in the other path is used as the first image. The first image is used for viewing by the user, and the second image acquisition direction corresponding to the first image changes with the change of the flight speed direction.

[0220] Step 503: output the first image; wherein, when the change in the direction of the flight speed is always within the viewing angle of the same first image acquisition device, the first image is always from the same first image acquisition device.

[0221] Unlike the above-described embodiment, in steps 501-503, the image information captured by the aircraft's image acquisition device for detecting surrounding obstacles is transmitted in two separate channels. One channel is used to detect obstacles around the aircraft, enabling the aircraft to autonomously avoid obstacles based on this image information. The other channel is used for user viewing, allowing the user to perform corresponding control operations based on the environmental image in the direction of flight speed, thus avoiding safety issues that might arise if autonomous obstacle avoidance fails. Furthermore, the image information from the obstacle avoidance image acquisition device can be used both for autonomous obstacle avoidance by the aircraft and for user viewing, effectively improving image information utilization, eliminating the need for additional image acquisition devices, and reducing costs.

[0222] In some embodiments, in order to facilitate the user to observe the environmental image in the direction of flight speed, the first image is the image information captured by the first image acquisition device and stabilized based on the current attitude information of the aircraft. This can make the image viewed by the user a more stable image, thereby improving the user's viewing experience.

[0223] The flight speed direction may be located within the viewing angle of the first image acquisition device and may change within the viewing angle of the first image acquisition device.

[0224] Specifically, in some embodiments, the flight speed direction can roughly coincide with the second image acquisition direction, and the second image acquisition direction can be located in the middle of the viewing angle of the first image. In this way, the virtual optical axis of the first image imaging can always coincide with the flight speed direction, so that the user can observe a more comprehensive image around the aircraft along the flight speed direction, and the image can change in real time as the flight speed direction changes, thereby achieving a smooth transition during the image change process.

[0225] In some embodiments, the first image may be obtained by subjecting image information captured by the first image acquisition device to stabilization processing. The above method may further include: obtaining current attitude information of the aircraft; wherein the first image is obtained by subjecting the image information captured by the first image acquisition device to stabilization based on the current attitude information of the aircraft. Specifically, the stabilization method may be to crop the image information captured by the first image acquisition device based on the current attitude information of the aircraft to obtain the first image with a relatively stable perspective.

[0226] The specific implementation of the above steps 501-503 can also refer to the explanation of the above embodiment, and will not be repeated here to avoid repetition.

[0227] For the parts not mentioned in the embodiments of the present application, reference can be made to the relevant introduction of the embodiments described in the aforementioned steps 101-103. Without conflict, they can all be regarded as optional features of the embodiments of the present application and will not be described in detail here.

[0228] The following embodiments of the present application are described in detail with reference to the accompanying drawings. Unless otherwise specified, the following embodiments and features thereof may be combined with each other.

[0229] Please refer to Figure 14, which is a schematic block diagram of the structure of an auxiliary image processing device provided in an embodiment of the present application. The auxiliary image processing device is applied to the aforementioned aircraft 100 or control terminal 200. The control terminal can be integrated with the aforementioned aircraft 100 or independently provided and communicatively connected to the aircraft 100. The aforementioned control method can also be applied to the control terminal.

[0230] As shown in FIG14 , the auxiliary image processing device 1400 includes a processor 1401 and a memory 1402 . The processor 1401 and the memory 1402 are connected via a bus 1403 , which is, for example, an I 2 C (Inter-Integrated Circuit) bus.

[0231] Specifically, the processor 1401 may be a micro-controller unit (MCU), a central processing unit (CPU), or a digital signal processor (DSP).

[0232] Specifically, the memory 1402 may be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a mobile hard disk.

[0233] The processor 1401 is configured to run a computer program stored in the memory 1402 and implement at least one of the method steps described in steps 101-104, 201-203, 301-302, 401-402, and 501-503 when executing the computer program.

[0234] It should be noted that those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process of the auxiliary image processing device described above can refer to the corresponding process in the aforementioned control method embodiment and will not be repeated here.

[0235] The embodiment of the present application further provides an aircraft, including the auxiliary image processing device. The embodiment of the present application further provides a control terminal, including the auxiliary image processing device.

[0236] The present application also provides a computer-readable storage medium storing a computer program. The computer program includes program instructions, and the processor executes the program instructions to implement at least one of the method steps described in the above embodiment, including steps 101-104, 201-203, 301-302, 401-402, and 501-503.

[0237] The computer-readable storage medium may be an internal storage unit of the aircraft described in any of the aforementioned embodiments, such as a hard disk or memory of the aircraft. The computer-readable storage medium may also be an external storage device of the aircraft, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash memory card, etc., equipped on the aircraft.

[0238] It should be understood that the terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0239] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0240] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present application, and such modifications or substitutions should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for processing an auxiliary image, characterized in that: include: Get the current flight speed and direction of the aircraft; In response to a situation where the current flight speed direction of the aircraft is inconsistent with the first image acquisition direction of the aircraft, acquiring image information of a first perspective acquired by an image acquisition device of the aircraft in a second image acquisition direction, wherein the second image acquisition direction is associated with the current flight speed direction of the aircraft and is different from the first image acquisition direction; Determining image information of a second perspective from the image information of the first perspective based on the current flight speed direction of the aircraft; and The image information of the second perspective is output, wherein the second perspective is included in the first perspective, and the image information of the second perspective is used to obtain the environmental conditions of the current flight speed direction of the aircraft.

2. A method for processing auxiliary images, characterized in that: include: Acquire composite image information of a first perspective acquired by at least two image acquisition devices of the aircraft; Based on the current flight speed direction of the aircraft, determine image information of a second perspective from the synthetic image information of the first perspective, wherein the second perspective is included in the first perspective; The image information of the second perspective is output, wherein the image information of the second perspective is used to obtain the environmental conditions of the current flight speed direction of the aircraft.

3. The method according to claim 1 or 2, characterized in that: The flight speed direction is used to indicate the direction in which the fuselage position of the aircraft changes.

4. The method according to claim 3, characterized in that The flight speed direction includes a horizontal component direction and a vertical component direction, and determining the image information of the second perspective from the image information of the first perspective based on the flight speed direction of the aircraft includes: Based on the horizontal component direction and the vertical component direction of the flight speed direction of the aircraft, the image information of the second perspective is determined from the image information of the first perspective.

5. The method according to claim 4, characterized in that The horizontal component direction includes any one of front, back, left, and right.

6. The method according to claim 4, characterized in that The vertical component direction includes up or down.

7. The method according to claim 4, characterized in that The image information of the second viewing angle includes image information acquired by at least two image acquisition devices, and acquisition directions of the at least two image acquisition devices correspond to the horizontal component direction and the vertical component direction, respectively.

8. The method according to claim 1 or 2, characterized in that: The flight speed direction is within the viewing angle of the image acquisition device.

9. The method according to claim 1 or 2, characterized in that: The flight speed direction is located in the middle position of the second viewing angle.

10. The method according to claim 1 or 2, characterized in that: The method further comprises: Based on the change in the direction of the flight speed, the second viewing angle is changed.

11. The method according to claim 10, characterized in that The changing of the second viewing angle based on the change in the flight speed direction includes at least one of the following: Based on the change in the direction of the flight speed, changing the image acquisition device corresponding to the second viewing angle; Based on the change in the direction of the flight speed, changing the image acquisition direction of the image acquisition device corresponding to the second viewing angle; Based on the change in the direction of the flight speed, the size of the second viewing angle is changed.

12. The method according to claim 10, characterized in that Changing the second viewing angle based on the change in the flight speed direction includes: When the flight speed direction changes from the first direction to the second direction, the image acquisition device is controlled to change from a viewing angle corresponding to the first direction to a viewing angle corresponding to the second direction.

13. The method according to claim 12, characterized in that When the flight speed direction changes from the first direction to the second direction, the second viewing angle corresponding to the aircraft moving along the second direction changes at least along the pitch axis direction of the aircraft and / or along the height direction of the aircraft compared to when the aircraft moves along the first direction, wherein one of the first direction and the second direction includes only a horizontal component direction, and the other includes a horizontal component direction and a vertical component direction, or the vertical component directions of the first direction and the second direction are different.

14. The method according to claim 12, characterized in that The controlling the image acquisition device to change the viewing angle corresponding to the first direction to the viewing angle corresponding to the second direction comprises: Acquire image information of a viewing angle between a viewing angle corresponding to the first direction and a viewing angle corresponding to the second direction; When the image acquisition device changes from a viewing angle corresponding to a first direction to a viewing angle corresponding to a second direction, image information of a viewing angle between the viewing angle corresponding to the first direction and the viewing angle corresponding to the second direction is output.

15. The method according to claim 10, characterized in that The changing the second viewing angle based on the change in the flight speed direction includes: When the direction of the flight speed changes and the speed magnitude is greater than a preset threshold, the second viewing angle is changed.

16. The method according to claim 10, characterized in that The changing the second viewing angle based on the change in the flight speed direction includes: When the flight speed direction changes and the duration of maintaining the changed flight speed direction is greater than a preset threshold, the second viewing angle is changed.

17. The method according to claim 10, characterized in that The changing the second viewing angle based on the change in the flight speed direction includes: During the movement of the aircraft based on the preset trajectory, the change in the direction of the flight speed predicted based on the preset trajectory; Before the flight speed direction changes, the second viewing angle is changed based on the predicted flight speed direction.

18. The method according to claim 17, characterized in that The changing the second viewing angle based on the change in the flight speed direction includes: When the flight speed changes from containing only horizontal components or vertical components to containing both horizontal and vertical components, the aircraft is controlled to obtain image information of the second perspective by one sensor instead of obtaining image information of the second perspective by at least two sensors.

19. The method according to claim 18, characterized in that The at least two sensors are used to obtain image information along a horizontal direction and image information along a vertical direction respectively.

20. The method according to claim 10, characterized in that The changing the second viewing angle based on the change in the flight speed direction includes: When the change in the flight speed direction is less than a preset threshold, the image acquisition device for acquiring the image information of the second perspective is changed, and / or the image acquisition device for acquiring the image information of the second perspective is changed. gesture to change the second perspective; When the change in the flight speed direction is greater than or equal to a preset threshold, the image acquisition device for acquiring the image information of the second viewing angle is changed.

21. The method according to claim 1 or 2, characterized in that: The image information of the second perspective includes image information acquired by one or more image acquisition devices of the aircraft.

22. The method according to claim 21, characterized in that The determining the image information of the second perspective from the image information of the first perspective based on the flight speed direction of the aircraft includes: When the second viewing angle is greater than the viewing angle corresponding to the image acquisition device in the aircraft, image information of the second viewing angle is determined based on image information acquired by at least two image acquisition devices of the aircraft.

23. The method according to claim 22, characterized in that The one or more image acquisition devices of the aircraft are determined based on the flight speed direction.

24. The method according to claim 23, characterized in that The angle between the optical axis of the one or more image acquisition devices and the flight speed direction is smaller than the angle between the optical axis of other image acquisition devices of the aircraft and the flight speed direction.

25. The method according to claim 1 or 2, characterized in that The image information of the first viewing angle is obtained by synthesizing image information of multiple image acquisition devices with different viewing angles.

26. The method according to claim 25, characterized in that The image information of the second viewing angle is obtained by cutting the image information of the first viewing angle.

27. The method according to claim 25, characterized in that The image information of the second perspective is obtained by locally rendering the image information of the first perspective.

28. The method according to claim 1 or 2, characterized in that The image acquisition device is also used to output image information for autonomous obstacle avoidance.

29. The method according to claim 28, characterized in that The image information of the second perspective is obtained by stabilizing the image information used for autonomous obstacle avoidance.

30. The method according to claim 29, characterized in that The method further comprises: Acquiring attitude information of the aircraft; Based on the posture information, stabilization processing is performed on the image information of the second perspective.

31. The method according to claim 1 or 2, characterized in that: The method further comprises: Acquiring image information from a third perspective through an image acquisition device of the aircraft; Outputting the image information of the third perspective; The third viewing angle is determined based on the shooting direction.

32. The method according to claim 31, characterized in that The image information of the third viewing angle is obtained from the image information of the first viewing angle.

33. The method according to claim 31, characterized in that The image information of the third perspective is acquired based on a shooting device of the aircraft.

34. The method according to any one of claims 31, characterized in that The image information of the third viewing angle and the image information of the second viewing angle are transmitted through a single image.

35. The method according to claim 1 or 2, characterized in that When the aircraft is in the process of returning, the image information of the second perspective further includes at least one of the following: Image information below the aircraft; image information in the direction of the return point.

36. The method according to claim 1 or 2, characterized in that When the aircraft is in the process of landing, the image information of the second perspective includes image information of at least one direction of the aircraft's surrounding viewing direction and / or image information below the aircraft.

37. The method according to claim 1 or 2, characterized in that When the aircraft is flying around the target object, the image information of the second perspective also includes image information perpendicular to the flight speed direction of the aircraft and / or image information at a fixed angle to the flight speed direction.

38. The method according to claim 1 or 2, characterized in that When the aircraft is tracking a target object, the image information of the second perspective includes image information along a flight speed direction of the aircraft and / or image information in a direction toward the target object.

39. The method according to claim 1 or 2, characterized in that The image information of the second viewing angle is grayscale image information.

40. The method according to claim 31, characterized in that The outputting the image information of the second viewing angle includes: A first image corresponding to the image information of the second viewing angle and a second image corresponding to the image information of the third viewing angle are displayed.

41. The method according to claim 40, characterized in that The displaying of the first image corresponding to the image information of the second viewing angle and the second image corresponding to the image information of the third viewing angle includes: displaying the first image and the second image on the same interface, and / or, The first image and the second image are displayed on different interfaces.

42. The method according to claim 41, characterized in that When the first image and the second image are displayed on the same interface, the first image occupies a partial area of ​​the second image, or the second image occupies a partial area of ​​the first image.

43. The method according to claim 41, characterized in that The method further comprises: receiving a first input from a user; In response to the first input, the first image and the second image are switched for display.

44. The method according to claim 41, characterized in that Switching and displaying the first image and the second image includes: Switching the display ratio of the first image and the second image in the same interface; or, Switch the display interface to display the first image or the second image.

45. The method according to claim 40, characterized in that The method further comprises: Based on the shooting direction of the aircraft and the flying speed direction, the display mode of the first image and the second image is changed.

46. ​​The method according to claim 45, characterized in that When the shooting direction is consistent with the flying speed direction, the second image is displayed preferentially.

47. The method according to claim 45, characterized in that When the shooting direction is inconsistent with the flying speed direction, the first image is displayed preferentially.

48. The method according to claim 40, characterized in that The method further comprises: receiving a second input of the first image by a user; In response to the second input, the viewing angle of the first image is controlled to remain unchanged.

49. The method according to claim 40, characterized in that The method further comprises: receiving a third input from a user regarding the first image; In response to being triggered by the third input, changing the viewing angle of the first image; In response to the end of the third input, the viewing angle of the first image is restored.

50. The method according to claim 40, characterized in that The method further comprises: The first image is displayed based on obstacle information of the aircraft along the flight speed direction.

51. The method according to claim 50, characterized in that The method further comprises: When the distance between the aircraft and the obstacle along the viewing direction of the first image is less than a preset threshold, prompt information is displayed on the first image.

52. A method for processing an auxiliary image, characterized in that: include: Get the current flight speed and direction of the aircraft; When the first image acquisition direction of the aircraft is inconsistent with the current flight speed direction, a first image in a second image acquisition direction is output, wherein the second image acquisition direction is different from the first image acquisition direction, and the second image acquisition direction changes with the change of the flight speed direction of the aircraft.

53. A method for processing auxiliary images, characterized in that: include: Get the current flight speed and direction of the aircraft; Based on the current flight speed direction of the aircraft, a first image in a second image acquisition direction and a second image in a first image acquisition direction are output, wherein the second image acquisition direction changes with the change of the flight speed direction of the aircraft, the change of the first image acquisition direction is decoupled from the flight speed direction of the aircraft, and the first image is used to assist obstacle avoidance.

54. A method for processing an auxiliary image, characterized in that: include: Acquire image information collected by a plurality of first image collection devices of the aircraft, wherein the plurality of first image collection devices are used to sense obstacles around the aircraft; The image information collected by the first image collection device for obstacle avoidance in the current flight speed direction of the aircraft is divided into two transmission paths, wherein the image information transmitted in one path is used to sense obstacles around the aircraft so that the aircraft can autonomously avoid obstacles according to the image information, and at least part of the image information transmitted in the other path is used as a first image, wherein the first image is used for viewing by a user, and a second image collection direction corresponding to the first image changes with the change of the flight speed direction; Output the first image; wherein, when the change in the direction of the flight speed is always within the viewing angle of the same first image acquisition device, the first image always comes from the same first image acquisition device.

55. The method according to any one of claims 52 to 54, characterized in that The second image acquisition direction is determined based on a horizontal component direction and a vertical component direction of a flight speed direction of the aircraft.

56. The method according to any one of claims 52 to 54, characterized in that The flight speed direction is used to indicate the direction in which the fuselage position of the aircraft changes.

57. The method according to claim 56, characterized in that The flight speed direction includes a component along the horizontal direction of the aircraft and a component along the vertical direction of the aircraft.

58. The method according to claim 57, characterized in that The flight speed direction includes a horizontal component direction and a vertical component direction.

59. The method according to claim 58, characterized in that The first image includes image information in a horizontal component direction along the flight speed direction and image information in a vertical component direction along the flight speed direction.

60. The method of claim 57, wherein: The horizontal component direction includes at least one of front, back, left, and right.

61. The method according to claim 57, characterized in that The vertical component direction includes up and / or down.

62. The method according to claim 53, characterized in that The aircraft includes a first image acquisition device and a second image acquisition device, the first image is acquired based on the first image acquisition device, the second image is acquired based on the second image acquisition device, and the viewing angles of the first image acquisition device and the second image acquisition device are different.

63. The method according to claim 62, characterized in that The first image acquisition device is used to perceive the surrounding environment of the aircraft.

64. The method according to claim 63, characterized in that The second image acquisition device is used to obtain image information of the shooting device carried by the aircraft.

65. The method according to claim 62, characterized in that The flight speed direction is located within the viewing angle of the first image acquisition device and can change within the viewing angle of the first image acquisition device.

66. The method according to claim 62, characterized in that The flight speed direction roughly coincides with the second image acquisition direction, wherein the second image acquisition direction is located in the middle of the viewing angle of the first image.

67. The method according to any one of claims 52 to 54, characterized in that The first image is obtained based on image information having a first viewing angle, wherein the first image has a second viewing angle, and the second viewing angle is included in the first viewing angle.

68. The method according to any one of claims 52 to 54, characterized in that The first image is a local image obtained by synthesizing and processing image information of at least two image acquisition devices.

69. The method according to any one of claims 52 to 54, characterized in that The method further comprises: The viewing angle of the first image is changed based on the change in the flight speed direction of the aircraft.

70. The method according to claim 69, characterized in that Changing the viewing angle of the first image based on a change in the flight speed direction of the aircraft includes: Based on the change in the flight speed direction of the aircraft, changing the image acquisition direction of the image acquisition device corresponding to the first image; Based on the change in the flight speed direction of the aircraft, changing the image acquisition device corresponding to the first image; The viewing angle of the first image is changed based on the change in the flight speed direction of the aircraft.

71. The method according to claim 69, characterized in that The changing the viewing angle of the first image based on the change in the flight speed direction of the aircraft includes: When the flight speed direction changes and the flight speed is greater than a preset threshold, the viewing angle of the first image is changed.

72. The method of claim 69, wherein: When the direction of the flight speed changes and the duration of the change is greater than a preset threshold, the viewing angle of the first image is changed.

73. The method according to claim 69, characterized in that During operation based on the preset trajectory, a speed change is predicted based on the preset trajectory, and the viewing angle of the first image is changed before the speed changes.

74. The method according to claim 69, characterized in that When the flight speed changes from a first direction to a second direction, the viewing angle of the first image is controlled to change from a viewing angle corresponding to the first direction to a viewing angle corresponding to the second direction.

75. The method according to claim 74, characterized in that When the flight speed direction changes from the first direction to the second direction, the viewing angle of the first image when the aircraft moves along the second direction changes at least along the vertical direction of the aircraft compared to the viewing angle when the aircraft moves along the first direction, wherein one of the first direction and the second direction includes only a horizontal component direction, and the other includes a horizontal component direction and a vertical component direction, or the vertical component directions of the first direction and the second direction are different.

76. The method of claim 74, wherein: Controlling the viewing angle of the first image to change from the viewing angle corresponding to the first direction to the viewing angle corresponding to the second direction, comprising: Image information between the viewing angle corresponding to the first direction and the viewing angle corresponding to the second direction is displayed.

77. The method of claim 53, wherein: The outputting the first image in the second image acquisition direction and the second image in the first image acquisition direction comprises: displaying the first image and the second image on the same interface, and / or, The first image and the second image are displayed on different interfaces.

78. The method according to claim 77, characterized in that When the first image and the second image are displayed on the same interface, the first image occupies a partial area of ​​the second image, or the second image occupies a partial area of ​​the first image.

79. The method according to claim 77, characterized in that The method further comprises: receiving a first input from a user; In response to the first input, the first image and the second image are switched for display.

80. The method according to claim 79, characterized in that Switching and displaying the first image and the second image includes: Switching the display ratio of the first image and the second image in the same interface; or, Switch the display interface to display the first image or the second image.

81. The method of claim 77, wherein: The method further comprises: Based on the shooting direction of the aircraft and the flying speed direction, the display mode of the first image and the second image is changed.

82. The method according to claim 81, characterized in that When the shooting direction is consistent with the flying speed direction, the second image is displayed preferentially.

83. The method according to claim 81, characterized in that When the shooting direction is inconsistent with the flying speed direction, the first image is displayed preferentially.

84. The method according to any one of claims 52-54, characterized in that The method further comprises: receiving a second input of the first image by a user; In response to the second input, the viewing angle of the first image is controlled to remain unchanged.

85. The method according to any one of claims 52 to 54, characterized in that The method further comprises: receiving a third input from a user regarding the first image; In response to being triggered by the third input, changing the viewing angle of the first image; In response to the end of the third input, the viewing angle of the first image is restored.

86. The method according to any one of claims 52 to 54, characterized in that The method further comprises: The first image is displayed based on obstacle information of the aircraft along the flight speed direction.

87. The method according to claim 86, characterized in that The method further comprises: When the distance between the aircraft and the obstacle along the viewing direction of the first image is less than a preset threshold Next, a prompt message is displayed on the first image.

88. The method according to claim 53, characterized in that The first image and the second image are received through a transmission channel.

89. The method according to claim 53, characterized in that The method further comprises: A third image is displayed which is different from the flight speed direction and the shooting direction of the aircraft.

90. The method according to claim 89, characterized in that The viewing angle of the third image is determined based on the attitude of the control terminal of the aircraft; or, The viewing angle of the third image is determined based on a preset route direction of the aircraft.

91. The method according to any one of claims 52 to 54, characterized in that When the aircraft is in the process of returning home, the first image also includes image information below the aircraft and / or image information in the direction of the return point.

92. The method according to any one of claims 52 to 54, characterized in that When the aircraft is in the process of landing, the first image further includes image information of at least one direction of the aircraft's surrounding direction and / or image information below the aircraft.

93. The method according to any one of claims 52-54, characterized in that When the aircraft is flying around the target object, the first image further includes image information at a fixed angle to the flight speed direction, or image information perpendicular to the flight speed direction of the aircraft.

94. The method according to any one of claims 52-54, characterized in that During the process of the aircraft tracking the target object, the first image further includes image information along the flight speed direction of the aircraft and / or image information in the direction of the target object.

95. The method according to claim 54, characterized in that The flight speed direction is located within the viewing angle of the first image acquisition device and can change within the viewing angle of the first image acquisition device.

96. The method according to claim 95, characterized in that The flight speed direction roughly coincides with the second image acquisition direction, wherein the second image acquisition direction is located in the middle of the viewing angle of the first image.

97. The method according to claim 52 or 54, characterized in that The method further includes: outputting a second image, where the second image is used for viewing by a user, and the first image acquisition direction corresponding to the second image is the shooting direction of the main camera of the aircraft.

98. The method according to claim 97, characterized in that The outputting the first image comprises: displaying the first image and the second image on the same interface, and / or, The first image and the second image are displayed on different interfaces.

99. The method according to claim 98, characterized in that When the first image and the second image are displayed on the same interface, the first image occupies a partial area of ​​the second image, or the second image occupies a partial area of ​​the first image.

100. The method according to claim 98, characterized in that The method further comprises: receiving a first input from a user; In response to the first input, the first image and the second image are switched for display.

101. The method according to claim 100, characterized in that Switching and displaying the first image and the second image includes: Switching the display ratio of the first image and the second image in the same interface; or, Switch the display interface to display the first image or the second image.

102. The method according to claim 97, characterized in that The method further comprises: Based on the shooting direction of the aircraft and the flying speed direction, the display mode of the first image and the second image is changed.

103. The method according to claim 102, characterized in that When the shooting direction is consistent with the flying speed direction, the second image is displayed preferentially.

104. The method according to claim 102, characterized in that When the shooting direction is inconsistent with the flying speed direction, the first image is displayed preferentially.

105. The method according to claim 97, characterized in that The first image and the second image are received through a transmission channel.

106. The method according to claim 97, characterized in that The method further comprises: A third image is displayed which is different from the flight speed direction and the shooting direction of the aircraft.

107. The method according to claim 106, characterized in that The viewing angle of the third image is determined based on the attitude of the control terminal of the aircraft; or, The viewing angle of the third image is determined based on a preset route direction of the aircraft.

108. The method according to claim 97, characterized in that The method also includes: receiving the first image and the second image through a transmission channel.

109. The method according to claim 54, characterized in that The method further comprises: Acquire the current attitude information of the aircraft; wherein the first image is an image obtained after stabilization of the image information acquired by the first image acquisition device based on the current attitude information of the aircraft.

110. An auxiliary image processing device, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the method steps as described in any one of claims 1-109 when executing the computer program.

111. An aircraft, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the method steps as described in any one of claims 1-109 when executing the computer program.

112. A control terminal, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the method steps as described in any one of claims 1-109 when executing the computer program.

113. A readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, enables the processor to implement the method steps according to any one of claims 1 to 109.

Images