Path planning method and apparatus, control method and apparatus, control terminal, and storage medium

Abstract

A path planning method, comprising: determining a target operation area (S101); automatically generating, on the basis of a preset condition, a virtual operation area in a real-time view-finding picture of a movable platform (100), and determining, on the basis of the preset condition, geographical location information corresponding to the virtual operation area in a real three-dimensional space (S102); adjusting the virtual operation area on the basis of the target operation area (S103); and on the basis of the adjusted virtual operation area, planning an operation path (S104). The path planning method can improve the efficiency of path planning.

Description

Path planning method, control method, device, control terminal and storage medium Technical Field

[0001] The present application relates to the field of path planning technology, and in particular to a path planning method, control method, device, control terminal and storage medium. Background Art

[0002] Before conducting inspections or surveying, a mobile platform must plan a path for the target area of ​​the target object to ensure safety and accuracy. However, current methods for planning paths within target areas, particularly those for facade work areas, primarily rely on the mobile platform marking points to determine the target area's geographic location. This process, which requires the mobile platform to move to each point to collect the corresponding geographic location information, is cumbersome and complex, resulting in lengthy path planning times and a poor user experience.

[0003] Summary of the Invention

[0004] Based on this, the embodiments of the present application provide a path planning method, a control method, a device, a control terminal and a storage medium, which aim to improve the planning efficiency of the operation path to enhance the user experience.

[0005] In a first aspect, an embodiment of the present application provides a path planning method, including:

[0006] Determine the target operating area;

[0007] Automatically generating a virtual work area on a real-time view screen of a mobile platform according to preset conditions, wherein geographical location information corresponding to the virtual work area in real three-dimensional space is determined based on the preset conditions;

[0008] Adjusting the virtual operating area according to the target operating area; and

[0009] Plan the operation path based on the adjusted virtual operation area.

[0010] The entire implementation process of the path planning method provided in the first aspect does not require determining the target work area based on the movable platform, which reduces the complexity of path planning and improves the planning efficiency of the work path. Moreover, since the virtual work area is adjusted based on the target work area, the virtual work area can be adjusted according to user needs. For example, the virtual work area can be adjusted to ensure that the generated virtual work area fits the target work area. Therefore, planning the work path based on the virtual work area can ensure the matching degree between the work path and the target work area, thereby ensuring the work effect.

[0011] In a second aspect, an embodiment of the present application further provides a path planning method, including:

[0012] Determine the target operating area;

[0013] determining a distance between the target operating area and the movable platform;

[0014] generating a virtual work area on a real-time viewing screen of the movable platform based on parameters related to the distance; and

[0015] Based on the virtual operation area, an operation path is planned.

[0016] The entire implementation process of the path planning method provided in the second aspect does not require determining the target work area based on the marking of the movable platform, which reduces the complexity of path planning and improves the planning efficiency of the work path. In addition, since the virtual work area is generated in the real-time view screen based on parameters related to the distance of the target work area relative to the movable platform, the consistency of the generated virtual work area and the target work area can be guaranteed. Therefore, the work path is planned according to the virtual work area, which can ensure the matching degree between the work path and the target work area, thereby ensuring the work effect when using the work path for work, and the user experience is better.

[0017] In a third aspect, an embodiment of the present application further provides a path planning method for a mobile platform, comprising:

[0018] Display the three-dimensional shape of the target object in the image;

[0019] In response to a target surface determination operation, determining a target surface of the target object from the three-dimensional shape of the target object;

[0020] Automatically generate a reference surface based on the target surface; and

[0021] Based on the reference surface, a working path of the movable platform is generated.

[0022] The entire implementation process of the path planning method provided by the third aspect does not require determining the target operating area based on the marking of the movable platform, which reduces the complexity of path planning and improves the path planning efficiency of the movable platform. Moreover, since the reference surface is generated based on the target surface of the target object, the consistency between the generated reference surface and the target surface is guaranteed. Therefore, based on the reference surface, the operating path of the movable platform is generated, which can ensure the matching degree between the operating path and the target surface, thereby ensuring the operating effect when using the operating path for operation, and the user experience is better.

[0023] In a fourth aspect, an embodiment of the present application further provides a method for controlling a movable platform, comprising:

[0024] In response to a user's operation, controlling the movable platform to move to the vicinity of a target object, wherein the movable platform is equipped with an image sensor and a ranging sensor;

[0025] displaying an image of a target surface including the target object acquired by the image sensor;

[0026] determining a reference surface in the image based at least on the sensing information of the ranging sensor, wherein position information of the reference surface is associated with position information of the target surface;

[0027] In response to a user's adjustment operation on the reference surface, adjusting the reference surface in the image to obtain an adjusted reference surface;

[0028] generating an operating path of the movable platform based on the adjusted reference surface; and

[0029] The movable platform is controlled to move along the working path.

[0030] The entire implementation process of the control method provided in the fourth aspect does not require determining the measurement area based on the marking of points on the movable platform, thereby reducing the complexity of path planning and improving the efficiency of operation path planning. Furthermore, the movable platform can be controlled to perform operations immediately after the path is planned, eliminating the need for waiting and improving the real-time performance of the operation. Furthermore, by determining a reference surface in the image using at least the sensing information of the ranging sensor, the positional relationship between the reference surface and the target surface of the target object in the image can be well displayed, making it easy for the user to adjust the reference surface so that the adjusted reference surface is consistent with the target object. Therefore, based on the adjusted reference surface, the operation path of the movable platform is generated, ensuring the matching degree between the operation path and the target surface of the target object, thereby ensuring the operation effect when the movable platform uses the operation path.

[0031] In the fifth aspect, an embodiment of the present application also provides a path planning device, which includes a memory and a processor, the memory being used to store a computer program, and the processor being used to execute the computer program and, when executing the computer program, implement the path planning method as described in the first aspect, the second aspect, or the third aspect.

[0032] In the sixth aspect, an embodiment of the present application further provides a control device, which includes a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to execute the computer program and implement the control method described in the fourth aspect when executing the computer program.

[0033] In the seventh aspect, an embodiment of the present application also provides a control terminal, which includes a memory and a processor, the memory being used to store a computer program, and the processor being used to execute the computer program and, when executing the computer program, implement the path planning method as described in the first aspect, the second aspect or the third aspect or the control method as described in the fourth aspect.

[0034] In an eighth aspect, an embodiment of the present application further provides a storage medium for computer-readable storage, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the path planning method as described in the first aspect, the second aspect or the third aspect or the control method as described in the fourth aspect.

[0035] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0037] FIG1 is a schematic diagram of a scenario for implementing the path planning method provided in an embodiment of the present application;

[0038] FIG2 is a schematic flow chart of the steps of a path planning method provided in an embodiment of the present application;

[0039] FIG3 is an example diagram of a path planning interface in an embodiment of the present application;

[0040] FIG4 is another example diagram of a path planning interface in an embodiment of the present application;

[0041] FIG5 is a schematic flow chart of a sub-step of the path planning method in FIG2 ;

[0042] FIG6 is another example diagram of a path planning interface in an embodiment of the present application;

[0043] FIG7 is a schematic flow chart of another sub-step of the path planning method in FIG2 ;

[0044] FIG8 is another example diagram of a path planning interface in an embodiment of the present application;

[0045] FIG9 is a schematic flowchart of another sub-step of the path planning method in FIG2 ;

[0046] FIG10 is another example diagram of a path planning interface in an embodiment of the present application;

[0047] FIG11 is another example diagram of a path planning interface in an embodiment of the present application;

[0048] FIG12 is a schematic flow chart of the steps of another path planning method provided in an embodiment of the present application;

[0049] FIG13 is another example diagram of a path planning interface in an embodiment of the present application;

[0050] FIG14 is another example diagram of a path planning interface in an embodiment of the present application;

[0051] FIG15 is another example diagram of a path planning interface in an embodiment of the present application;

[0052] FIG16 is an example diagram of an overview diagram in an embodiment of the present application;

[0053] FIG17 is another example diagram of a path planning interface in an embodiment of the present application;

[0054] FIG18 is another example diagram of an overview diagram in an embodiment of the present application;

[0055] FIG19 is another example diagram of an overview diagram in an embodiment of the present application;

[0056] FIG20 is another example diagram of a path planning interface in an embodiment of the present application;

[0057] FIG21 is a schematic flow chart of the steps of another path planning method provided in an embodiment of the present application;

[0058] FIG22 is a schematic flow chart of the steps of another path planning method provided in an embodiment of the present application;

[0059] FIG23 is a schematic flow chart of the steps of another path planning method provided in an embodiment of the present application;

[0060] FIG24 is a schematic block diagram of the structure of a path planning device provided in an embodiment of the present application;

[0061] FIG25 is a schematic block diagram of the structure of a control device provided in an embodiment of the present application;

[0062] Figure 26 is a schematic block diagram of the structure of a control terminal provided in an embodiment of the present application. DETAILED DESCRIPTION

[0063] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0064] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined, or partially merged, so the actual execution order may vary depending on the actual situation.

[0065] The following describes some embodiments of the present application in detail with reference to the accompanying drawings. In the absence of conflict, the following embodiments and features therein may be combined with each other.

[0066] Please refer to FIG1 , which is a schematic diagram of a scenario for implementing the path planning method provided in an embodiment of the present application.

[0067] As shown in Figure 1, the movable platform 100 is communicatively connected to the control terminal 200, and the control terminal 200 is used to control the movable platform 100. Among them, the movable platform 100 includes a platform body 110, a power device 120 provided on the platform body 100, an image sensor 130 and a control system (not shown in Figure 1). The power device 120 is used to provide mobile power for the movable platform 100, the image sensor 130 is used to capture images, and the control system is used to control the movable platform. It should be noted that the movable platform 100 may not be provided with the power device 120, that is, the movable platform 100 may not move autonomously, but may be moved with the help of external forces (such as people or vehicles).

[0068] The movable platform 100 may include an aircraft, an unmanned vehicle, a mobile robot, a camera, or a mobile phone. Aircraft may include unmanned aerial vehicles (UAVs) and manned aerial vehicles. UAVs include rotary-wing UAVs, such as quad-rotor UAVs, hexacopter UAVs, and octo-rotor UAVs. They may also be fixed-wing UAVs, or a combination of rotary-wing and fixed-wing UAVs. UAVs can be categorized by application industry into agricultural UAVs, industrial UAVs, and aerial photography UAVs. The control terminal 200 includes remote controls, smartphones, tablet computers, laptop computers, and PCs.

[0069] In some embodiments, the power unit 120 may include one or more propellers 121, one or more motors 122 corresponding to the propellers, and one or more electronic speed controllers (ESCs). The motors 122 are connected between the ESCs and the propellers 121. The motors 122 and propellers 121 are mounted on the platform body 110 of the movable platform 100. The ESCs receive drive signals from a control system and provide current to the motors 122 based on the drive signals to control the speed of the motors 122. The motors 122 drive the propellers 121, thereby providing power for the movement of the movable platform 100. This power enables the movable platform 100 to achieve one or more degrees of freedom. In some embodiments, the movable platform 100 can rotate about one or more rotational axes. For example, the rotational axes may include roll, yaw, and pitch. It should be understood that the motors 122 may be either DC or AC motors. Furthermore, the motors 122 may be either brushless or brushed motors. In other embodiments, the power device 120 may also be a wheel device or an underwater drive device.

[0070] In some embodiments, the control system includes a control device and a sensing system. The sensing system is used to measure the posture information of the movable platform 100, that is, the position information and state information of the movable platform 100 in space, such as three-dimensional position, three-dimensional angle, three-dimensional velocity, three-dimensional acceleration, and three-dimensional angular velocity. The sensing system may include, for example, at least one of a gyroscope, an ultrasonic sensor, an electronic compass, an inertial measurement unit (IMU), a visual sensor, a global navigation satellite system, and a barometer. For example, the global navigation satellite system may be a global positioning system (GPS). The control device is used to control the movement of the movable platform 100. For example, the movement of the movable platform 100 may be controlled based on the posture information measured by the sensing system. It should be understood that the control device may control the movable platform 100 according to pre-programmed instructions.

[0071] In some embodiments, a path planning method includes: determining a target work area; automatically generating a virtual work area in a real-time view image of a movable platform according to preset conditions, wherein the geographical location information corresponding to the virtual work area in real three-dimensional space is determined based on the preset conditions; adjusting the virtual work area according to the target work area; and planning a work path based on the adjusted virtual work area. It should be noted that in some embodiments, the virtual work area is only presented in the real-time view image and does not necessarily have to be actually projected in real three-dimensional space. The virtual work area can be projected into the video stream using AR projection to achieve more intuitive interaction and display. In this embodiment, since the geographic location information of the virtual operation area can be determined based on preset conditions, the virtual operation area can be adjusted on this basis to obtain the geographic location information of the adjusted virtual operation area; the virtual operation area can be automatically generated after the target operation area is determined, and the operation path can be planned based on the virtual operation area. Therefore, there is no need to determine the target operation area based on the movable platform, which reduces the complexity of path planning and improves the efficiency of operation path planning. Moreover, since the virtual operation area is adjusted based on the target operation area, the virtual operation area can be adjusted according to user needs. For example, the virtual operation area can be adjusted to ensure that the generated virtual operation area is aligned or overlapped with the target operation area. Therefore, planning the operation path based on the virtual operation area can ensure the matching degree between the operation path and the target operation area, thereby ensuring the operation effect. It should be noted that the adjusted virtual operation area may not completely align with the target operation area, but may be partially aligned or overlapped, or even not aligned or overlapped. This is mainly adjusted according to user needs, such as the user's operation accuracy.

[0072] The planned operation path can be executed by the mobile platform 100 itself, or sent to other mobile platforms for execution.

[0073] In some embodiments, the path planning method can be applied to a control terminal 200, which is configured to be in communication with a mobile platform 100. The control terminal 200 displays a real-time view of the mobile platform's view through a display device 210. In other embodiments, the path planning method can also be applied to the mobile platform 100. The mobile platform 100 can have a built-in display module for displaying the real-time view. The mobile platform can include an aerial photography aircraft, a ground-based viewing device, a mobile scanning device, a mobile measuring device, a mobile mapping device, a camera, or a mobile phone.

[0074] In some embodiments, a path planning method includes: determining a target work area; determining a distance between the target work area and a movable platform; generating a virtual work area in a real-time view of the movable platform based on parameters related to the distance; and planning a work path based on the virtual work area. In this embodiment, a virtual work area can be generated in a real-time view of the movable platform based on parameters related to the distance, and the geographic location information of the generated virtual work area can be determined based on the parameters related to the distance. Since the relative positional relationship between the target work area and the movable platform is fully considered, the generated virtual work area can be close to the target work area. Therefore, the work path can be planned based on the virtual work area, and there is no need to determine the target work area based on the marking of the movable platform, thereby reducing the complexity of path planning and improving the efficiency of work path planning.

[0075] The planned operation path can be executed by the mobile platform 100 itself, or sent to other mobile platforms for execution.

[0076] In some embodiments, the path planning method can be applied to a control terminal 200, which is configured to be in communication with a mobile platform 100. The control terminal 200 displays a real-time view of the mobile platform's view through a display device 210. In other embodiments, the path planning method can also be applied to the mobile platform 100. The mobile platform 100 can have a built-in display module for displaying the real-time view. The mobile platform can include an aerial photography aircraft, a ground-based viewing device, a mobile scanning device, a mobile measuring device, a mobile mapping device, a camera, or a mobile phone.

[0077] In some embodiments, a path planning method for a movable platform includes: displaying a three-dimensional shape of a target object in an image; determining a target surface of the target object from the three-dimensional shape of the target object in response to a target surface determination operation; automatically generating a reference surface based on the target surface; and generating an operation path based on the reference surface. This embodiment can automatically generate a reference surface based on the target surface, and the geographic location information of the reference surface is close to the geographic location information of the target surface. Therefore, an operation path can be generated based on the reference surface, without the need to determine the target operation area based on the mobile platform marking, thereby reducing the complexity of path planning and improving the efficiency of path planning for the movable platform. Moreover, since the reference surface is generated based on the target surface of the target object, the operation path of the movable platform is generated based on the reference surface, which can ensure the matching degree between the operation path and the target surface, thereby ensuring the operation effect when using the operation path.

[0078] The path planning method can be applied to the control terminal 200, which displays an image through the display device 210. The image is a 2D image or a 3D image acquired by the image sensor 130; or, the image is a 2D image or a 3D image generated based on the sensing information of the ranging sensor. The image can be a real-time view screen, or a 2D model or a 3D model diagram. In other embodiments, the path planning method can also be applied to a mobile platform. The mobile platform can have its own display module for displaying the image. The planned operation path can be used for execution by the mobile platform 100 that implements the path planning, or it can be sent to other mobile platforms for execution.

[0079] In some embodiments, a method for controlling a movable platform includes: in response to a user's operation, controlling the movable platform 100 to move to the vicinity of a target object, wherein the movable platform 100 is equipped with an image sensor 130 and a ranging sensor (not shown in FIG1 ); displaying an image acquired by the image sensor 130 and including a target surface of the target object; determining a reference surface in the image based at least on sensing information from the ranging sensor, wherein position information of the reference surface is associated with position information of the target surface; in response to a user's adjustment operation on the reference surface, adjusting the reference surface in the image to obtain an adjusted reference surface; generating an operating path of the movable platform 100 based on the adjusted reference surface; and controlling the movable platform 100 to move along the operating path. This embodiment determines a reference surface in the image based at least on the sensing information of the ranging sensor. The geographic location information of the generated reference surface can be determined based on the sensing information. The location information of the reference surface is associated with the location information of the target surface, so that the generated reference surface can be close to the target surface. Therefore, the operation path can be planned based on the reference surface. This embodiment does not need to determine the target operation area based on the marking of the movable platform, which reduces the complexity of path planning and improves the path planning efficiency of the movable platform. In addition, the movable platform can be controlled to perform operations immediately after the path is planned without waiting, thereby improving the real-time performance of the operation.

[0080] The control method can be applied to the control terminal 200, which displays images through the display device 210, and can also be applied to the mobile platform 100, which can have its own display module for displaying images.

[0081] The following will describe in detail the path planning method or control method provided by the embodiments of the present application in conjunction with the scenario in Figure 1. It should be noted that the scenario in Figure 1 is only used to explain the path planning method or control method provided by the embodiments of the present application, but does not constitute a limitation on the application scenarios of the path planning method or control method provided by the embodiments of the present application.

[0082] Please refer to Figure 2, which is a flowchart illustrating the steps of a path planning method provided in an embodiment of the present application. This path planning method is applied to a control terminal configured to communicate with a mobile platform with a camera function. The mobile platform may include an aerial vehicle, a ground-based mobile platform, a mobile scanning device, a mobile measuring device, a mobile mapping device, a handheld gimbal camera, or a mobile phone.

[0083] As shown in FIG2 , the path planning method includes steps S101 to S104 .

[0084] Step S101: Determine the target operation area.

[0085] In this embodiment, the target operation area may be an area requiring surveying, aerial photography, spraying, sowing, placement, or cleaning. The target operation area may be a planar operation area, which may be a flat operation area, an inclined operation area, or a vertical operation area. For example, the target operation area may be farmland, the surface of a cliff, or one of the surfaces of a building. The flat operation area is generally parallel to the ground, water, or sea surface, meaning that the angle between the flat operation area and the ground, water, or sea surface is less than or equal to a first set angle. The inclined operation area is generally greater than the first set angle and less than or equal to a second set angle. The vertical operation area is generally perpendicular to the ground, water, or sea surface, meaning that the angle between the vertical operation area and the ground, water, or sea surface is greater than the second set angle and less than or equal to 90°. The first and second set angles can be set based on actual circumstances and are not specifically limited in this embodiment of the present invention. For example, the first set angle is 15° and the second set angle is 75°.

[0086] In some embodiments, a target work area is determined in a displayed image, wherein the displayed image may include a real-time view or a map. A user's selection of the target work area is received in the displayed image, or the path planning device automatically determines the target work area, without requiring the movable platform to mark the target work area.

[0087] In some embodiments, determining the target work area includes at least one of: obtaining a user's selection of the target work area, or automatically determining the target work area based on preset target work area determination conditions. Obtaining the user's selection of the target work area includes at least one of: obtaining a user's operation on a physical shortcut key; or obtaining a user's click or circling operation on the screen. The user's operation on the physical shortcut key can be performed by clicking or sliding the physical shortcut key when the target work area appears on the live view screen, or when a viewfinder or pre-selected box frames the target work area, thereby determining the target work area. The preset target work area determination conditions can be set by the user or based on actual circumstances, and are not specifically limited in this embodiment of the present invention. For example, if the preset target work area determination condition is to identify a building with a height greater than a set height and a tower shape, then when the movable platform identifies a building with a height greater than the set height and a tower shape, it can determine one of the surfaces of the identified building as the target work area. This embodiment provides multiple methods for determining the target work area to meet different scenarios and improve the user experience.

[0088] In some embodiments, when a user clicks on the screen, one of the surfaces of the clicked object on the screen is identified as the target operating area, or an area of ​​a preset size centered on the clicked position on the screen is identified as the target operating area. The preset size can be set based on actual conditions, and the embodiments of the present application do not specifically limit this. For example, the preset size is 2cm*2cm or 3cm*3cm, etc. When a user circles the screen, the circled area is used as the target operating area.

[0089] In some embodiments, the displayed real-time view screen is displayed with a pre-selection box, which is used to indicate that the area framed on the real-time view screen is the target operation area. When the user's operation on the physical shortcut key is obtained, the area currently framed by the pre-selection box on the real-time view screen is confirmed as the target operation area. Alternatively, when the user's click operation on the measurement area confirmation icon on the screen is obtained, the area currently framed by the pre-selection box on the real-time view screen is confirmed as the target operation area. For example, as shown in Figure 3, the displayed real-time view screen 10 is displayed with a pre-selection box 11 and a measurement area confirmation icon 12. At this time, if the user's click operation on the physical shortcut key or the measurement area confirmation icon 12 is obtained, the area currently framed by the pre-selection box 11 on the real-time view screen 10 is confirmed as the target operation area.

[0090] Step S102: automatically generating a virtual operation area in the real-time view image of the movable platform according to preset conditions, wherein geographical location information corresponding to the virtual operation area in the real three-dimensional space is determined based on the preset conditions.

[0091] In this embodiment, the virtual work area automatically generated in the live view is displayed superimposed on the live view. The live view is a real image, and the virtual work area is a virtual image. In the live view, the generated virtual work area can completely or partially overlap the target work area, or it can be located outside the target work area.

[0092] In some embodiments, the planning method further includes: in response to the target work area being determined, automatically generating a virtual work area based on preset conditions in the live view image of the movable platform. The determination of the target work area provides trigger information for generating the virtual work area, thereby enabling rapid generation of the virtual work area.

[0093] In a further embodiment, in response to the target work area being determined, a virtual work area is immediately generated.

[0094] In a further embodiment, in response to the viewing direction of the movable platform facing the target work area and in response to the target work area being determined, a virtual work area is generated.

[0095] For example, when the image sensor 130 is shooting at the target work area 30, the target work area 30 is displayed on the live view screen. For example, the pre-selection box 11 at least partially frames the target work area 30. In response to receiving the user's operation to determine the target work area 30, the virtual work area 20 is automatically generated on the live view screen. This embodiment can achieve the rapid generation of the virtual work area. It should be noted that the trigger condition for the generation of the virtual work area of ​​the present application can also be set to other conditions according to actual needs, for example, within a preset time after the target work area is determined or automatically generated upon receiving a preset operation.

[0096] In some embodiments, the preset condition may include: determining the position of the virtual working area based on the relative position relationship between the target working area and the movable platform.

[0097] In some embodiments, the position of the virtual work area is determined based on the relative position relationship between the target work area and the movable platform, including: determining the position of the virtual work area based on the distance between the target work area and the movable platform, so that the virtual work area is as close to the target work area as possible in the radial direction of the real three-dimensional space (referring to the direction of the distance between the target work area and the movable platform).

[0098] Specifically, the distance between the position corresponding to the virtual work area in the real three-dimensional space and the movable platform is equal to or approximately equal to the distance between the target work area and the movable platform, thereby ensuring that the virtual work area is as close as possible to the target work area in the real three-dimensional space; alternatively, there is a preset difference between the distance between the position corresponding to the virtual work area in the real three-dimensional space and the movable platform and the distance between the target work area and the movable platform. The preset difference can be set according to actual needs. It should be noted that the distance between the target work area and the movable platform can refer to the distance between the target work area and the movable platform body, or it can be the distance between the target work area and the load on the movable platform body (such as an image sensor or a ranging sensor, etc.). The distance between the target work area and the movable platform can be the distance between a representative point of the target work area (such as the point of the target work area corresponding to the center point of the pre-selected box) and the movable platform, or it can be the distance between multiple feature points on the target work area and the movable platform. When the distance is the distance between the representative point of the target work area (such as the point in the target work area corresponding to the center point of the pre-selected box) and the movable platform, what is determined is the geographic location information of the point in the virtual work area corresponding to the representative point, and based on the geometric information of the virtual work area, the geographic location information of the entire virtual work area can be determined.

[0099] In a further embodiment, the position of the virtual working area is determined based on the relative position relationship between the target working area and the movable platform, and the further step includes: determining the orientation of the virtual working area based on a fixed distance direction, so that the virtual working area is closer to the target working area in the circumferential direction surrounding the movable platform in the real three-dimensional space, wherein the fixed distance direction is the direction for measuring the distance between the movable platform and the target working area; or, determining the orientation of the virtual working area based on the framing direction of the movable platform, generally the framing direction is toward the target working area, and if the virtual working area is projected according to the framing direction, the virtual working area can also be made closer to the target working area.

[0100] It should be noted that the above-mentioned preset condition one: determining the position of the virtual work area based on the distance between the target work area and the movable platform, and preset condition two: determining the orientation of the virtual work area based on the fixed distance direction or determining the orientation of the virtual work area based on the framing direction of the movable platform, preset condition one and preset condition two can exist at the same time, or one of them can exist selectively. When the position of the virtual work area is determined based on the distance between the target work area and the movable platform, the orientation of the virtual work area can be set according to actual needs. When the orientation of the virtual work area is determined based on the fixed distance direction or the framing direction of the movable platform, the distance between the virtual work area and the movable platform can be determined according to actual needs.

[0101] In some embodiments, the projection direction of the virtual work area is consistent with the distance direction, or an angle is set between the projection direction of the virtual work area and the distance direction; or the projection direction of the virtual work area is consistent with the framing direction, or an angle is set between the projection direction of the virtual work area and the framing direction.

[0102] When the orientation of the virtual work area is related to the fixed distance direction, the fixed distance direction is aligned with the projection direction of the virtual work area, making the virtual work area more closely aligned with the target work area. Alternatively, an angle is defined between the fixed distance direction and the projection direction of the virtual work area. The size of this angle can be adjusted based on actual conditions.

[0103] When the orientation of the virtual work area is related to the framing direction, the framing direction is aligned with the projection direction of the virtual work area, bringing the virtual work area closer to the target work area. Alternatively, an angle is defined between the framing direction and the projection direction of the virtual work area. The size of the angle can be adjusted based on actual conditions.

[0104] In some embodiments, the viewing direction of the movable platform is consistent with the distance direction, that is, the distance direction can be determined based on the viewing direction of the movable platform. In a specific embodiment, the viewing direction of the movable platform can be directed toward the target operation area, and the distance direction at this time is the viewing direction of the movable platform. It should be noted that the viewing direction of the movable platform and the distance direction can also be provided with an angle, that is, they are not in the same direction. Determining the distance direction by the viewing direction realizes the function of automatically determining the distance measurement point on the target operation area. For example, the viewing direction will determine the viewing direction of the pre-selected frame, changing the position indicated by the center of the pre-selected frame, that is, the position indicated by the center of the pre-selected frame can represent the viewing direction. When the viewing direction is adjusted to face the target operation area, the point in the actual target operation area corresponding to the center of the pre-selected frame can be used as the distance measurement point. At this time, the measured distance can be the distance between the center / center of gravity of the movable platform body or the center / center of gravity of the image sensor and the distance measurement point of the above-mentioned actual target operation area. The distance direction is the direction in which the center / center of gravity of the movable platform body or the center / center of gravity of the image sensor points to the distance measurement point of the actual target operating area.

[0105] It should be noted that when the movable platform is a panoramic viewfinder, since there is no fixed shooting direction, the fixed distance direction can be determined as the placement direction of the virtual operation area.

[0106] Additionally, in some embodiments, determining the position of the virtual work area based on the relative positional relationship between the target work area and the movable platform may also include determining the position of the virtual work area based on the orientation of the target work area relative to the movable platform. That is, by obtaining the orientation of the target work area relative to the movable platform, the projection direction of the virtual work area is determined, and the specific projection distance of the virtual work area can be arbitrarily set according to actual needs. For example, when it is determined that the target work area is directly in front of the movable platform, the virtual work area is projected directly in front of the movable platform. When it is determined that the target work area is in front of the left side of the movable platform, the virtual work area is projected to the corresponding position in front of the left side of the movable platform.

[0107] The distance between the movable platform and the target operating area is measured by a ranging sensor, which may include at least one of the following: a time of flight (TOF) ranging sensor, a laser ranging sensor, an ultrasonic ranging sensor, an infrared ranging device, a radar device, etc.

[0108] In some embodiments, in response to determining the target operating area, the distance between the target operating area and the movable platform can be determined based on a pre-generated perception map and the movable platform's own geographic location information. The perception map is a pre-constructed visual map based on sensory data collected by the movable platform's visual sensors. The geographic location information of the target operating area can be obtained through the visual map. Based on the movable platform's own geographic location information and the geographic location information of the target operating area, the distance between the target operating area and the movable platform can be calculated. Using the pre-generated perception map and the movable platform's own geographic location information, the distance between the target operating area and the movable platform can be accurately and quickly determined.

[0109] In some embodiments, the displayed live view image shows the distance between the target work area and the movable platform. For example, as shown in FIG3 , the live view image displays distance indication information 13 and first prompt information 14. Distance indication information 13 is used to indicate the distance between the target work area and the movable platform. In FIG3 , the distance between the target work area and the movable platform is 61.9 meters. First prompt information 14 prompts the user to determine the target work area. If the distance between the target work area and the movable platform exceeds a preset distance range, a second prompt information is output. For example, the distance between the target work area and the movable platform displayed in the live view image and / or the display color of the pre-selected box are changed to a preset color to indicate to the user that the distance measurement failed. The preset distance range can be set based on actual conditions and is not specifically limited in this embodiment of the present application. For example, if the preset distance range is [4, 150], the second prompt information is output if the distance between the target work area and the movable platform is less than 4 meters or greater than 150 meters.

[0110] In some embodiments, the viewing direction of the movable platform is perpendicular or approximately perpendicular to the target work area, so that the virtual work area generated on the live view image is parallel or approximately parallel to the target work area within the live view image. The viewing direction of the movable platform can be the direction of the shooting optical axis of the image sensor on the movable platform. This embodiment can accurately determine the corresponding geographic location information of the virtual work area in real three-dimensional space by using preset conditions and the geographic location information of the movable platform itself.

[0111] It should be noted that in some embodiments, the framing direction is consistent with the direction of the shooting optical axis, while in other embodiments, such as in panoramic cameras, the framing direction and the shooting optical axis are not directly related. In some embodiments, the path planning method provided in embodiments of the present application, before step S101, further includes: obtaining a user adjustment instruction for adjusting the framing angle of the movable platform; and adjusting the framing angle of the movable platform based on the adjustment instruction so that the framing direction of the movable platform is perpendicular or approximately perpendicular to the target work area. Adjusting the framing angle of the movable platform may include at least one of the following: adjusting the position of the movable platform, adjusting the posture of the movable platform, or adjusting the posture of a gimbal of the movable platform, wherein the image sensor is connected to the gimbal. The gimbal may be a three-axis gimbal. In this embodiment, based on the user adjustment instruction, the framing angle of the movable platform is adjusted so that the framing direction of the movable platform is perpendicular or approximately perpendicular to the target work area, thereby ensuring that the virtual work area subsequently generated on the live view image is parallel or approximately parallel to the target work area within the live view image.

[0112] In some embodiments, the path planning method provided by the embodiment of the present application, before step S101, further includes: determining the angle of the target operating area based on a pre-generated perception map; automatically determining the viewing angle of the movable platform based on the angle of the target operating area, so that the viewing direction of the movable platform is perpendicular or approximately perpendicular to the target operating area. The angle of the target operating area can be the angle of the target operating area relative to the horizontal plane or relative to the movable platform, and the angle of the target operating area can be obtained through the pre-generated perception map. This embodiment determines the angle of the target operating area based on the perception map, and based on the angle of the target operating area, it can automatically determine the viewing angle of the movable platform under the condition that the viewing direction is perpendicular or approximately perpendicular to the target operating area, thereby reducing user participation and ensuring that the virtual operating area subsequently generated on the real-time view screen can be parallel or approximately parallel to the target operating area in the real-time view screen.

[0113] In some embodiments, step S102 specifically includes: generating a virtual work area in the real-time view screen based on a pre-selected box and preset conditions. In a further embodiment, in response to the viewing direction of the movable platform facing the target work area, and in response to the target work area being determined, a virtual work area can be generated in the real-time view screen based on the pre-selected box and preset conditions. For example, as shown in Figures 3 and 4, the real-time view screen 10 shown in Figure 3 displays a pre-selected box 11 and a measurement area confirmation icon 12. At this time, if the user clicks the physical shortcut key or the measurement area confirmation icon 12, the area currently selected by the pre-selected box 11 on the real-time view screen 10 is confirmed as the target work area. At this time, the virtual work area 20 in Figure 3 is generated in the real-time view screen 10 based on the pre-selected box 11 and the preset conditions.

[0114] The size of the virtual work area in the real-time view is determined based on a preselected box, and the position of the virtual work area is determined based on preset conditions. Specifically, the position of the virtual work area in real three-dimensional space is determined based on the preset conditions. This embodiment determines the size of the virtual work area in the real-time view based on the preselected box, thereby simplifying the determination of the virtual work area. During the actual planning process, it is only necessary to adjust the distance between the movable platform and the target work area so that the preselected box can at least partially cover the target work area. The size of the preselected box in real three-dimensional space is determined based on the measured distance. In other words, the larger the distance value, the larger the size of the preselected box in real three-dimensional space. Conversely, the smaller the distance value, the smaller the size of the preselected box in real three-dimensional space. By simply adjusting the movable platform so that the preselected box covers the target work area as much as possible, the generated virtual work area can be made to fit the target work area as closely as possible.

[0115] In some embodiments, the size of the pre-selection box in the real-time view screen is a preset size. The size of the pre-selection box can be set in advance according to actual needs. Combined with the release distance of the generated virtual work area and the preset size of the pre-selection box, the geographical location of the virtual work area can be determined more easily.

[0116] It should be noted that the preset conditions here may be consistent with the aforementioned preset conditions, that is, the preset conditions include determining the position of the virtual operating area based on the relative position relationship between the target operating area and the movable platform, which will not be elaborated here.

[0117] In some embodiments, as shown in FIG5 , step S102 specifically includes: sub-steps S1021 to S1022 .

[0118] Sub-step S1021 : extracting the contour of the target operating area in the real-time view image according to the determined target operating area.

[0119] For example, as shown in FIG6 , the target operation area is determined to be the surface area 15 of the building, so the contour line 16 of the surface area 15 can be extracted in the real-time view image 10. In this embodiment, a preset contour line extraction algorithm can be used to extract the contour line of the target operation area, or a pre-trained contour line extraction model can be used to extract the contour line of the target operation area. The preset contour line extraction algorithm can be an edge detection algorithm based on a binary image, a contour line extraction algorithm based on color segmentation, a contour line extraction algorithm based on morphological processing, a contour line extraction algorithm based on distance transformation, or a contour line extraction algorithm based on curvature. The pre-trained contour line extraction model can be a neural network model trained using a large number of sample images, in which the sample operation areas in the sample images are marked with contour lines.

[0120] Sub-step S1022: generating a virtual operation area in the live view image based on the area defined by the contour line and preset conditions.

[0121] In a further embodiment, in response to the viewing direction of the movable platform facing the target working area and in response to the target working area being determined, a virtual working area can be generated in the real-time viewing screen based on the area defined by the contour line and preset conditions.

[0122] This embodiment extracts the contour line of the target work area in the real-time view screen, and based on the area circled by the contour line in the real-time view screen and preset conditions, it can generate a virtual work area covering the target work area in the real-time view screen, thereby reducing the deviation between the virtual work area and the target work area, reducing the time required for subsequent adjustment of the virtual work area, and further improving the overall path planning efficiency.

[0123] In some embodiments, generating a virtual work area in the live view based on the area defined by the contour line and preset conditions may include: determining the size of the virtual work area in the live view based on the area defined by the contour line, and determining the position of the virtual work area based on the preset conditions. Specifically, determining the corresponding position of the virtual work area in real three-dimensional space based on the preset conditions. The preset conditions and the geographic location information of the movable platform itself can be used to accurately determine the geographic location information corresponding to the virtual work area in real three-dimensional space. This embodiment can accurately determine the size of the virtual work area in the live view based on the area defined by the contour line, and accurately determine the position of the virtual work area based on the preset conditions, so that the generated virtual work area can effectively cover the target work area in the live view.

[0124] It should be noted that the preset conditions here may be consistent with the aforementioned preset conditions, that is, the preset conditions include determining the position of the virtual working area based on the relative position relationship between the target working area and the movable platform, which will not be elaborated here.

[0125] In some embodiments, as shown in FIG7 , step S102 specifically includes: sub-steps S1023 to S1026 .

[0126] Sub-step S1023: determining a plurality of feature points according to the determined target operation area.

[0127] Among them, the step of determining multiple feature points includes: obtaining the user's operation of determining multiple feature points, such as the user clicking on the real-time view screen; or, automatically extracting multiple feature points. In this embodiment, a preset feature point extraction algorithm can be used to automatically extract feature points, and extract multiple feature points of the target operation area in the real-time view screen. Among them, the preset feature point extraction algorithm can be set based on actual conditions, and the embodiment of the present application does not make specific limitations on this. For example, the preset feature point extraction algorithm includes at least one of the following: SIFT (Scale-Invariant Feature Transform) algorithm, SURF (Speeded-Up Robust Feature) algorithm, ORB (Oriented FAST and Rotated BRIEF) algorithm, Haris corner detection algorithm, and Hessian-Laplace algorithm.

[0128] In some embodiments, the multiple feature points determined include at least the endpoints of the boundary line of the target work area. For example, as shown in FIG6 , the target work area determined is the surface area 15 of a building. Therefore, multiple feature points of the building surface area 15 can be extracted in the live view image 10, and the extracted multiple feature points include the endpoints of the boundary line of the building surface area 15. Because the multiple feature points extracted include at least the endpoints of the boundary line of the target work area, the virtual work area subsequently generated based on the multiple feature points can effectively cover the target work area in the live view image.

[0129] Sub-step S1024: Obtain the relative position relationship between each feature point and the movable platform.

[0130] In this embodiment, the relative positional relationship between the feature point and the movable platform includes the distance between the feature point and the movable platform, as well as the measurement direction when measuring the distance between the feature point and the movable platform. In the aforementioned description of the preset conditions, the preset conditions include: determining the position of the virtual work area based on the relative positional relationship between the target work area and the movable platform. In this embodiment, it means that based on the distance between each feature point in the target work area and the movable platform, the distance between each corresponding point in the virtual work area and the movable platform is correspondingly determined, and the orientation of each corresponding point in the virtual work area is also determined based on the distance direction of each feature point.

[0131] It should be noted that the preset conditions in this article include: determining the position of the virtual work area based on the relative position relationship between the target work area and the movable platform, wherein the distance in the relative position relationship can be the distance between a representative point of the target work area (such as the point in the target work area corresponding to the center point of the pre-selected box) and the movable platform, or it can be the distance between multiple feature points on the target work area and the movable platform. When the distance in the relative position relationship is the distance between a representative point of the target work area (such as the point in the target work area corresponding to the center point of the pre-selected box) and the movable platform, what is determined is the geographic location information of the point in the virtual work area corresponding to the representative point, and based on the geometric information of the virtual work area, the geographic location information of the entire virtual work area can be determined.

[0132] Sub-step S1025: Determine the geographical location information of each feature point based on the geographical location information and relative position relationship of the movable platform.

[0133] In this embodiment, the geographic location information of the movable platform can be determined by the positioning device in the movable platform. The geographic location information of the movable platform and the relative position relationship between the feature point and the movable platform can be used to accurately determine the geographic location information of the feature point.

[0134] Sub-step S1026: Based on the geographic location information of each feature point, generate and display a virtual operation area covering the target operation area.

[0135] In a further embodiment, in response to the viewing direction of the movable platform facing the target working area and in response to the target working area being determined, a virtual working area covering the target working area is generated and displayed based on the geographic location information of each feature point.

[0136] In this embodiment, the geographic location information of multiple feature points can be determined through the relative position relationship between multiple feature points in the target operation area and the movable platform and the geographic location information of the movable platform. Based on the geographic location information of the feature points, the position of the feature points can be determined, so that a virtual operation area covering the target operation area can be generated and displayed.

[0137] Furthermore, based on the geographic location information of each feature point, the step of generating a virtual operation area includes: superimposing a point marker on the corresponding feature point according to the geographic location information of the feature point; specifically, based on the geographic location information of the feature point, the position of the feature point in the real-time view image can be determined, so that the point marker is superimposed on the corresponding position, so as to realize superimposing the point marker on the corresponding feature point. Among them, the specific display style of the point marker can be set based on the actual situation, and the embodiment of the present application does not make specific restrictions on this. For example, the point marker is a white dot, a red dot or a black dot of a preset pixel size. By connecting multiple point markers according to the actual relative position relationship between multiple feature points, a virtual operation area covering the target operation area can be obtained, which reduces the deviation between the virtual operation area and the target operation area, reduces the time required for subsequent adjustment of the virtual operation area, and further improves the overall path planning efficiency.

[0138] For example, as shown in Figure 8, in the real-time view screen 10, the point mark 17 is superimposed on the 8 feature points of the surface area 15 of the building, thereby obtaining 8 point marks 17. These 8 point marks 17 are connected according to the actual relative position relationship between the 8 feature points of the surface area 15 of the building, and a virtual working area 18 covering the surface area 15 of the building can be obtained.

[0139] Step S103: Adjust the virtual operating area according to the target operating area.

[0140] In some embodiments, the virtual operating area is adjusted according to the target operating area so that the adjusted virtual operating area coincides with the target operating area or the deviation between the adjusted virtual operating area and the target operating area is less than a preset value. In other embodiments, the virtual operating area can also be adjusted according to the actual needs of the user.

[0141] In this embodiment, in step S103, the user may visually determine whether the adjusted virtual operating area and the target operating area overlap or whether the deviation between the virtual operating area and the target operating area is less than a preset value, or the user may automatically determine whether the adjusted virtual operating area and the target operating area overlap or whether the deviation between the virtual operating area and the target operating area is less than a preset value. This embodiment of the application does not specifically limit this. Wherein, the overlap between the adjusted virtual operating area and the target operating area means that the adjusted virtual operating area fits the target operating area perfectly, and the outlines of the adjusted virtual operating area and the target operating area overlap. In some scenarios, it is impossible to accurately know the relative position relationship between the target operation area and the virtual operation area, such as errors in the measured distance, which makes it impossible to directly obtain the geographic location information of the target operation area. Therefore, through the pre-generated virtual operation area, the geographic location information of the virtual operation area can be accurately known, and through adjustment operations, the change in geographic location information corresponding to the adjustment operation can also be determined according to preset rules. Therefore, through the virtual operation area and adjustment operations, when the user visually determines in the real-time view screen that the virtual operation area coincides with the target operation area or the deviation between the virtual operation area and the target operation area is less than the preset value, the geographic location information of the target operation area can be inferred.

[0142] The step of adjusting the virtual operating area according to the target operating area includes: adjusting the virtual operating area based on an acquired user area adjustment operation, wherein the user adjusts the virtual operating area according to the target operating area; or automatically adjusting the virtual operating area according to the target operating area and a preset area adjustment algorithm. The following embodiments will be described using the example of adjusting the virtual operating area by a user's manual area adjustment operation. It will be appreciated by those skilled in the art that most of the following embodiments are also applicable to automatic adjustment.

[0143] In some embodiments, step S103 may include at least a first operation and / or a second operation, wherein the first operation is used to adjust at least one of the size, shape, position, and orientation of the virtual operating area, and the second operation is used to adjust the distance between the virtual operating area and the target operating area and / or the angle of the virtual operating area relative to the target operating area. In this embodiment, by adjusting at least one of the size, shape, position, and orientation of the virtual operating area, the outline of the adjusted virtual operating area and the outline of the target operating area are made as close as possible; by adjusting the distance between the virtual operating area and the target operating area and / or the angle of the virtual operating area relative to the target operating area, the adjusted virtual operating area and the target operating area can be made to fit as closely as possible. In general, by performing at least one of the first operation and the second operation, the adjusted virtual operating area and the target operating area are overlapped or the deviation between the adjusted virtual operating area and the target operating area is less than a preset value.

[0144] In some embodiments, the deviation between the virtual operating area and the target operating area is less than a preset value, including at least one of the following: the distance between the virtual operating area and the target operating area in the Z-axis direction of the preset three-dimensional coordinate system is less than a first preset value, the plane formed by the X-axis and Y-axis of the preset three-dimensional coordinate system is parallel to the virtual operating area, and the Z-axis is perpendicular to the virtual operating area; the tilt angle between the virtual operating area and the target operating area is less than a second preset value, and the tilt angle is the angle at which the virtual operating area is tilted relative to the target operating area around the X-axis or Y-axis; the distance between the outline of the virtual operating area and the outline of the target operating area is less than a third preset value; the deviation angle between the virtual operating area and the target operating area in the Z-axis direction is less than a fourth preset value. Among them, the first preset value, the second preset value, the third preset value, and the fourth preset value can be set based on actual conditions, and the embodiments of the present application do not specifically limit this. For example, the first preset value is 2 meters, the second preset value is 5°, the third preset value is 1.8 meters, and the fourth preset value is 8°.

[0145] In some embodiments, the path planning method provided by embodiments of the present application further includes: displaying a preset three-dimensional coordinate system on the virtual work area. The preset three-dimensional coordinate system also adaptively adjusts in response to area adjustment operations. By displaying the preset three-dimensional coordinate system on the virtual work area, this embodiment facilitates the user to determine the deviation between the virtual work area and the target work area using the preset three-dimensional coordinate system, providing a reference for the user to adjust the virtual work area, thereby improving the convenience and accuracy of adjusting the virtual work area.

[0146] In some embodiments, as shown in FIG9 , step S103 may include: sub-steps S1031 to S1032 .

[0147] Sub-step S1031 : adjusting the movable platform so that the viewing direction of the movable platform is switched to other angles of the target operation area, and the real-time viewing screen presents the target operation area and the virtual operation area at other angles.

[0148] In this embodiment, the other angles are different from the viewing angles of the movable platform when generating the virtual work area. The position, posture, and / or position or posture of the movable platform body and / or the image sensor can be adjusted automatically or manually by the user to change the viewing direction of the movable platform, so that the viewing direction of the movable platform is switched to other angles of the target work area. In this way, the real-time viewing screen presents the target work area and the virtual work area at other angles, which facilitates the subsequent determination of whether the virtual work area and the target work area overlap or whether the deviation between the virtual work area and the target work area is less than a preset value. For example, the real-time viewing screen 10 shown in Figure 4 presents the virtual work area 20 and the target work area 30 in the front. After adjusting the movable platform, the real-time viewing screen 10 shown in Figure 10 can be obtained. The real-time viewing screen 10 shown in Figure 10 presents the virtual work area 20 and the target work area 30 in the side.

[0149] Sub-step S1032: adjusting the virtual operating area according to the target operating area at other angles and the virtual operating area.

[0150] In this embodiment, the target operation area and the virtual operation area at other angles are used to supplement different perspectives to determine whether the virtual operation area overlaps with the target operation area or whether the deviation between the virtual operation area and the target operation area is less than a preset value. Then, when it is determined that the virtual operation area does not overlap with the target operation area or the deviation between the virtual operation area and the target operation area is greater than or equal to the preset value, the virtual operation area is adjusted automatically or manually by the user to obtain the adjusted virtual operation area, thereby improving the adjustment accuracy and convenience of the virtual operation area.

[0151] In this embodiment, the virtual operating area can be adjusted by obtaining the target operating area and the virtual operating area at the angle of the virtual operating area, and the adjustment operation may include the above-mentioned first operation and second operation; at the same time, the virtual operating area can be adjusted by changing the viewing angle so that the real-time viewing screen presents the target operating area and the virtual operating area at different angles, and the adjustment operation may also include the above-mentioned first operation and second operation.

[0152] In some embodiments, the target operation area is a planar area, and adjusting the movable platform so that the viewing direction of the movable platform is switched to other angles of the target operation area may include: adjusting the movable platform so that the viewing direction of the movable platform is switched to the side of the target operation area, and the real-time viewing screen includes the side of the target operation area and the side of the virtual operation area; adjusting the virtual operation area according to the target operation area and the virtual operation area at other angles, and obtaining the adjusted virtual operation area may include: when the viewing direction of the movable platform is switched to the side of the target operation area, performing a second operation to adjust the distance between the virtual operation area and the target operation area and / or adjusting the angle of the virtual operation area relative to the target operation area, so that the adjusted virtual operation area coincides with the target operation area or the deviation between the adjusted virtual operation area and the target operation area is less than a preset value. In this embodiment, when the target work area is a planar area, the viewing direction of the movable platform is changed so that the real-time viewing image includes the side surface of the target work area and the side surface of the virtual work area. The side surface of the target work area and the side surface of the virtual work area can accurately determine whether the virtual work area and the target work area overlap or whether the deviation between the virtual work area and the target work area is less than a preset value. If it is determined that the virtual work area and the target work area do not overlap or the deviation between the virtual work area and the target work area is greater than or equal to the preset value, the virtual work area is automatically or manually adjusted by the user, thereby improving the accuracy and convenience of adjusting the virtual work area. It should be noted that the first operation can also be performed to adjust the virtual work area from this side perspective.

[0153] For example, the real-time view screen 10 shown in FIG4 presents the virtual working area 20 on the front and the target working area 30 on the front. After adjusting the movable platform, the real-time view screen 10 shown in FIG10 can be obtained. The real-time view screen 10 shown in FIG10 presents the virtual working area 20 on the side and the target working area 30 on the side.

[0154] In some embodiments, before step S101, the path planning method provided in embodiments of the present application further includes: adjusting the movable platform so that the viewing direction of the movable platform is the front of the target work area, and the real-time view image at least includes a portion of the front of the target work area. The position and posture of the movable platform body and / or the position or posture of the image sensor can be adjusted automatically or manually by the user to change the viewing direction of the movable platform so that the viewing direction of the movable platform is switched to the front of the target work area.

[0155] In some embodiments, adjusting the virtual work area according to the target work area includes: adjusting at least one of the size, shape, position and orientation of the virtual work area according to the target work area on the front. Wherein, when the viewing direction of the movable platform is the front of the target work area, a first operation can be performed to adjust at least one of the size, shape, position and orientation of the virtual work area. Since the front of the target work area can fully display the size, shape, position and orientation of the target work area, when the front of the target work area is included in the real-time view screen and the virtual work area is superimposed on the real-time view screen, at least one of the size, shape, position and orientation of the virtual work area can be adjusted so that the adjusted virtual work area can cover the target work area. It should be noted that, under this frontal perspective, the second operation can also be performed to adjust the virtual work area.

[0156] In some embodiments, the step of adjusting the virtual operating area according to the target operating area includes: adjusting the virtual operating area based on an acquired area adjustment operation of the user, wherein the user adjusts the virtual operating area according to the target operating area.

[0157] In some embodiments, the virtual operation area includes at least one adjustment point located on the boundary of the virtual operation area, and the area adjustment operation includes a user dragging or clicking operation on the adjustment point. The user dragging or clicking operation on the adjustment point is used to adjust the size and / or shape of the virtual operation area. In other embodiments, the virtual operation area includes multiple endpoint adding buttons and / or endpoint deleting buttons, and the area adjustment operation includes a user dragging or clicking operation on the endpoint adding buttons and / or endpoint deleting buttons. The user dragging or clicking operation on the endpoint adding buttons is used to add endpoints of the virtual operation area to adjust the size and / or shape of the virtual operation area, and the user dragging or clicking operation on the endpoint deleting buttons is used to delete endpoints of the virtual operation area to adjust the size and / or shape of the virtual operation area. In other embodiments, at least one endpoint adding button and / or endpoint deleting button is displayed on each boundary of the virtual operation area.

[0158] For example, as shown in FIG11 , a virtual work area 20 is superimposed on the live view screen 10. The virtual work area 20 includes four adjustment points 21 and four endpoint addition buttons 22 located on its boundaries. Furthermore, an endpoint addition button 22 is displayed on each boundary of the virtual work area 20. The user can drag or click the adjustment points 21 to adjust the size and / or shape of the virtual work area. The user can also drag or click the endpoint addition button 22 to add endpoints to the virtual work area, thereby adjusting the size and / or shape of the virtual work area.

[0159] In some embodiments, in step S103: the virtual operating area can also be automatically adjusted according to the target operating area and the preset area adjustment algorithm, so that the adjusted virtual operating area coincides with the target operating area or the deviation between the adjusted virtual operating area and the target operating area is less than a preset value. Among them, the preset area adjustment algorithm can be set based on actual conditions, and the embodiments of the present application do not specifically limit this. For example, the deviation between the target operating area and the virtual operating area is automatically determined, and according to the deviation between the target operating area and the virtual operating area, the size, shape, position, orientation of the virtual operating area, the distance between the virtual operating area and the target operating area and / or the angle of the virtual operating area relative to the target operating area are automatically adjusted, so that the adjusted virtual operating area coincides with the target operating area or the deviation between the adjusted virtual operating area and the target operating area is less than a preset value.

[0160] In some embodiments, the step of acquiring a user's area adjustment operation includes at least one of the following: acquiring the area adjustment operation in a live view image; acquiring the area adjustment operation in an overview image, wherein the overview image displays the target work area and the virtual work area framed at a preset angle by the movable platform. The overview image is a captured image. This embodiment enables adjustment of the virtual work area based on the live view image and / or the overview image, allowing the user to use the appropriate adjustment method according to actual needs, thereby improving the user experience.

[0161] In some embodiments, when a region adjustment operation is performed on the live view screen, the virtual work area in the overview map also changes in response to the region adjustment operation; and / or when a region adjustment operation is performed on the overview map, the virtual work area in the live view screen also changes in response to the region adjustment operation. This embodiment enables synchronous adjustment of the virtual work area in the live view screen and the virtual work area in the overview map, ensuring consistency between the virtual work area in the live view screen and the virtual work area in the overview map.

[0162] In some embodiments, the path planning method provided by the embodiments of the present application further includes: automatically generating an overview map in response to the generation of the virtual work area. This embodiment automatically generates the overview map at the same time as the virtual work area is generated, thereby enabling the overview map to be generated quickly and improving the user experience.

[0163] In some embodiments, the route planning method provided in embodiments of the present application further includes: obtaining an overview map in response to a user's photo operation. In this embodiment, the overview map obtained in response to the user's photo operation better meets the user's needs, avoiding situations where an automatically generated overview map fails to meet the user's needs. The overview map can be regenerated by taking a photo, thereby improving the user experience.

[0164] In some embodiments, the path planning method provided in embodiments of the present application further includes: automatically storing the overview map after obtaining it, or storing the overview map in response to a user's request to store the overview map. By storing the overview map, this embodiment enables the virtual work area to be adjusted using the stored overview map even if the communication connection with the mobile platform is disconnected, thereby improving the user experience.

[0165] In some embodiments, a perception map is superimposed on the live view display. The perception map includes a perception model of the target work area, which is used to provide a reference basis for area adjustment operations. In this embodiment, by superimposing the perception map on the live view display of the mobile platform, and the perception map includes a perception model of the target work area, the displayed perception model can be used to determine how to adjust the virtual work area so that the adjusted virtual work area coincides with the target work area or the deviation between the adjusted virtual work area and the target work area is less than a preset value, thereby improving the efficiency and accuracy of virtual work area adjustment.

[0166] In some embodiments, in step S103 , the adjusted geographical location information of the virtual operating area may be determined based on the obtained geographical location information of the virtual operating area and the corresponding change in geographical location information when the virtual operating area is adjusted.

[0167] In some embodiments, the geographical location information of the virtual operation area can be determined based on the geographical location information of the movable platform and the above-mentioned preset conditions. The geographical location information of the movable platform can be determined based on its own positioning module.

[0168] In some embodiments, the amount of change in geographic location information corresponding to adjustments to the virtual work area can be determined based on preset rules and automatically acquired. For example, when adjusting and dragging the boundaries of the virtual work area on the display interface, the distance moved on the display interface can be calculated as the amount of change in real three-dimensional space based on a preset mapping algorithm. When the virtual work area is moved laterally, the distance moved on the display interface can be calculated as the amount of change in real three-dimensional space based on a preset mapping algorithm. Similarly, the amount of change in geographic location information associated with adjustments to the virtual work area on the display interface can be automatically acquired.

[0169] Step S104: planning an operation path based on the adjusted virtual operation area.

[0170] In some embodiments, when the virtual operating area is adjusted to coincide with the target operating area, the geographic location information of the adjusted virtual operating area can substantially represent the geographic location information of the target operating area. Therefore, planning an operating path based on the adjusted virtual operating area information is equivalent to planning an operating path based on the acquired geographic location information of the target operating area. In other embodiments, the virtual operating area can be adjusted based on actual needs, and the adjusted virtual operating area may not coincide with the target operating area.

[0171] In some embodiments, the planned operation path includes a framing path, an inspection path, a spreading path, a spraying path, a cleaning path, etc.

[0172] In some embodiments, planning the work path based on the adjusted virtual work area may include generating a work path based on the adjusted virtual work area. The adjustment of the virtual work area is determined to be complete in response to a user confirming the virtual work area. This embodiment generates a work path for the movable platform based on the adjusted virtual work area after the adjustment of the virtual work area, thereby ensuring a good match between the work path and the target work area, thereby ensuring a satisfactory work effect when the movable platform uses the work path.

[0173] In some embodiments, planning an operation path based on the adjusted virtual operation area may include: generating an initial operation path based on the virtual operation area, and adaptively adjusting the initial operation path based on the adjustment to the virtual operation area, thereby obtaining an operation path planned based on the adjusted virtual operation area. It is understandable that each time the virtual operation area is adjusted, the initial operation path is adaptively adjusted, and after the virtual operation area has been adjusted, the initial operation path is also adjusted. This embodiment generates an initial operation path based on the generated virtual operation area while generating the virtual operation area, and then adaptively adjusts the initial operation path while adjusting the virtual operation area. In this way, after the virtual operation area has been adjusted, the initial operation path is also adjusted, thereby obtaining a final operation path, thereby improving the real-time performance of path planning.

[0174] In some embodiments, based on the adjusted virtual work area, planning the work path may include: automatically generating a "bow" or "Z" shaped work path based on the virtual work area, the work parameters set by the user, and the preset planning rules. Among them, the preset planning rules can be set based on actual conditions, and the embodiments of the present application do not specifically limit this. The work parameters set by the user include surveying and mapping parameters, aerial photography parameters, spraying parameters, sowing parameters or scanning parameters, etc. For example, the movable platform is an aerial photography aircraft, and the work parameters set by the user include the ground sampling distance, the distance of the path relative to the target work area, the route speed, the route direction, the altitude mode, the safe take-off height, the heading overlap rate, the lateral overlap rate and the route starting point, etc. The automatic work path of this embodiment based on the adjusted virtual work area, the work parameters set by the user, and the preset planning rules is more in line with the user's needs, and can ensure the matching degree between the work path and the target work area, thereby ensuring the work effect when the movable platform uses the work path to perform operations, and the user experience is better.

[0175] In some embodiments, the movable platform can operate according to the operation path, that is, the planning device and the device that performs the operation can be the same device, thereby realizing the integration of planning operations; or, the generated operation path can be sent to another movable platform, so that the operation path can be planned in advance, and other operation equipment does not need to plan the operation path again.

[0176] In some embodiments, after generating a work path, the path planning method provided in embodiments of the present application further includes: determining whether there is a path segment in the work path that collides with an obstacle; and if so, generating a prompt message. The prompt message may include marking the path segment in the work path that collides with an obstacle. By marking the path segment in the work path that collides with an obstacle, this embodiment allows the user to be aware of the collision risk of the path segment, making it easier for the user to adjust the path.

[0177] Furthermore, it is possible to determine whether there is a path segment in the operation path that collides with an obstacle based on the pre-generated perception map.

[0178] In some embodiments, the path planning method provided by embodiments of the present application further includes: obtaining a user's operation parameter adjustment operation in a real-time view image and / or obtaining a user's operation parameter adjustment operation in an overview image; and adaptively adjusting the operation path based on the operation parameter adjustment operation. This embodiment can adjust operation parameters based on the real-time view image and / or the overview image, allowing the user to select an appropriate adjustment method to adjust operation parameters according to actual needs, thereby improving the user experience.

[0179] In some embodiments, after generating the work path, the path planning method provided in embodiments of the present application further includes: adjusting the work path based on a pre-generated perception map, and displaying the adjusted work path, so that the adjusted work path does not have any path segments that collide with obstacles. Alternatively, in response to a user's adjustment operation on the work path, adjusting the work path and displaying the adjusted work path, so that the adjusted work path does not have any path segments that collide with obstacles.

[0180] In some embodiments, the path planning method can be applied to a control terminal 200, which is configured to be in communication with a mobile platform 100. The control terminal 200 displays a real-time view of the mobile platform's view through a display device 210. In other embodiments, the path planning method can also be applied to the mobile platform 100. The mobile platform 100 can have a built-in display module for displaying the real-time view. The mobile platform can include an aerial photography aircraft, a ground-based viewing device, a mobile scanning device, a mobile measuring device, a mobile mapping device, a camera, or a mobile phone.

[0181] Please refer to FIG. 12 , which is a schematic flow chart of the steps of another path planning method provided in an embodiment of the present application.

[0182] As shown in FIG12 , the path planning method includes steps S201 to S205 .

[0183] Step S201: displaying a path planning interface, where the path planning interface includes a real-time viewing screen of the movable platform and a pre-selection box displayed on the real-time viewing screen.

[0184] For example, the path planning interface shown in FIG13 includes a live view screen 10 of the movable platform and a preselection box 11 displayed on the live view screen. The area currently selected by the preselection box 11 is the target operation area. The size of the preselection box 13 on the live view screen 10 is a preset size.

[0185] Step S202: Acquire a framing adjustment operation, and adjust the framing direction or size according to the framing adjustment operation so that the pre-selected frame at least partially covers the target operation area.

[0186] In this embodiment, the framing adjustment operation includes at least one of the following: a position adjustment operation of the movable platform body; an attitude adjustment operation of the movable platform body; an attitude adjustment operation of the image sensor carried by the movable platform; and an adjustment operation of the focus of the image sensor carried by the movable platform. The position adjustment operation of the movable platform body is used to adjust the position of the movable platform, the attitude adjustment operation of the movable platform body is used to adjust the attitude of the movable platform, the attitude adjustment operation of the image sensor carried by the movable platform is used to adjust the attitude of the image sensor, and the focus adjustment operation of the image sensor carried by the movable platform is used to adjust the focus of the image sensor.

[0187] In some embodiments, adjusting the framing direction and size based on a framing adjustment operation so that the pre-selected box at least partially covers the target work area within the live view image may include: adjusting the framing angle of the movable platform based on the framing adjustment operation so that the pre-selected box at least partially covers the target work area within the live view image, and the framing direction of the movable platform is perpendicular or approximately perpendicular to the target work area. In this embodiment, by adjusting the framing angle of the movable platform so that the framing direction of the movable platform is perpendicular or approximately perpendicular to the target work area, it is ensured that the virtual work area subsequently generated on the live view image is parallel or approximately parallel to the target work area within the live view image, and that the pre-selected box at least partially covers the target work area within the live view image.

[0188] Step S203: obtaining a user click operation on a physical shortcut key or a measurement area confirmation icon displayed on a path planning interface, and generating a virtual operation area in a real-time view screen according to a pre-selected box and preset conditions in response to the click operation.

[0189] For example, as shown in Figure 13, if a user clicks a physical shortcut key or a measurement area confirmation icon 12, the area currently selected by the pre-selection box 11 on the live view screen 10 is confirmed as the target work area, and a virtual work area is generated on the live view screen based on the pre-selection box 11 and preset conditions. As shown in Figure 14, a virtual work area 20 generated based on the pre-selection box 11 and preset conditions is superimposed on the live view screen 10.

[0190] In some embodiments, generating a virtual work area in the live view based on a preselected box and preset conditions may include: determining the size of the virtual work area in the live view based on the preselected box, and determining the position of the virtual work area based on the preset conditions. Specifically, determining the position of the virtual work area in real three-dimensional space based on the preset conditions. This embodiment determines the size of the virtual work area in the live view based on the preselected box, thereby simplifying the determination of the virtual work area. During the actual planning process, only the distance between the movable platform and the target work area needs to be adjusted so that the preselected box at least partially covers the target work area. The corresponding size of the preselected box in real three-dimensional space is determined based on the measured distance. In other words, the larger the distance value, the larger the size of the preselected box in real three-dimensional space. Conversely, the smaller the distance value, the smaller the size of the preselected box in real three-dimensional space. Simply adjusting the movable platform so that the preselected box covers the target work area as much as possible can ensure that the generated virtual work area fits the target work area as closely as possible.

[0191] In some embodiments, the size of the pre-selection box in the real-time view screen is a preset size. The size of the pre-selection box can be set in advance according to actual needs. Combined with the release distance of the generated virtual work area and the preset size of the pre-selection box, the geographical location of the virtual work area can be determined more easily.

[0192] It should be noted that the preset conditions here may be consistent with the aforementioned preset conditions, that is, the preset conditions include determining the position of the virtual operating area based on the relative position relationship between the target operating area and the movable platform, which will not be elaborated here.

[0193] Step S204: Adjust the virtual operation area based on the acquired area adjustment operation of the user.

[0194] In this embodiment, the virtual operating area is adjusted based on the user's area adjustment operation so that the adjusted virtual operating area coincides with the target operating area or the deviation between the adjusted virtual operating area and the target operating area is less than a preset value. In other embodiments, the virtual operating area can also be adjusted based on the user's actual needs.

[0195] In some scenarios, it is impossible to accurately know the relative position relationship between the target operation area and the virtual operation area, such as errors in the measured distance, which makes it impossible to directly obtain the geographic location information of the target operation area. Therefore, through the pre-generated virtual operation area, the geographic location information of the virtual operation area can be accurately known, and through adjustment operations, the change in geographic location information corresponding to the adjustment operation can also be determined according to preset rules. Therefore, through the virtual operation area and adjustment operations, when the user visually determines in the real-time view screen that the virtual operation area coincides with the target operation area or the deviation between the virtual operation area and the target operation area is less than the preset value, the geographic location information of the target operation area can be inferred.

[0196] In this embodiment, in step S204, the user can visually determine whether the adjusted virtual operating area overlaps with the target operating area or whether the deviation between the virtual operating area and the target operating area is less than a preset value. The fact that the adjusted virtual operating area overlaps with the target operating area means that the adjusted virtual operating area fits the target operating area perfectly and the outlines of the adjusted virtual operating area and the target operating area overlap.

[0197] In some embodiments, step S204 may include at least a first operation and / or a second operation, wherein the first operation is used to adjust at least one of the size, shape, position, and orientation of the virtual operating area, and the second operation is used to adjust the distance between the virtual operating area and the target operating area and / or the angle of the virtual operating area relative to the target operating area. In this embodiment, by adjusting at least one of the size, shape, position, and orientation of the virtual operating area, the outline of the adjusted virtual operating area and the outline of the target operating area are made as close as possible; by adjusting the distance between the virtual operating area and the target operating area and / or the angle of the virtual operating area relative to the target operating area, the adjusted virtual operating area and the target operating area can be made to fit as closely as possible. In general, by performing at least one of the first operation and the second operation, the adjusted virtual operating area and the target operating area are overlapped or the deviation between the adjusted virtual operating area and the target operating area is less than a preset value.

[0198] In some embodiments, when the viewing direction of the movable platform is the front of the target work area, at least one of the size, shape, position, and orientation of the virtual work area is adjusted based on the front target work area. It should be noted that a second operation can also be performed to adjust the virtual work area from this frontal perspective. For example, as shown in FIG14 , a virtual work area 20 is superimposed on the live view screen 10. The virtual work area 20 includes four adjustment points 21 and four endpoint addition buttons 22 located on its boundaries, and an endpoint addition button 22 is displayed on each boundary of the virtual work area 20. The area adjustment operation can include a user dragging or clicking an adjustment point 21, that is, a user dragging or clicking an adjustment point 21 can adjust the size and / or shape of the virtual work area. The area adjustment operation can also include a user dragging or clicking an endpoint addition button 22, that is, a user dragging or clicking an endpoint addition button 22 can add endpoints to the virtual work area, thereby adjusting the size and / or shape of the virtual work area.

[0199] In some embodiments, the step of acquiring a user's area adjustment operation includes at least one of the following: acquiring the area adjustment operation in a live view image; acquiring the area adjustment operation in an overview image, wherein the overview image displays the target work area and the virtual work area framed at a preset angle by the movable platform. The overview image is a captured image. This embodiment enables adjustment of the virtual work area based on the live view image and / or the overview image, allowing the user to use the appropriate adjustment method according to actual needs, thereby improving the user experience.

[0200] For example, as shown in FIG15 , the live view screen 10 also displays a measurement area adjustment pop-up window 23 and a measurement area adjustment button 24 for the virtual work area 20. Through the measurement area adjustment pop-up window 23, the user can adjust the distance between the virtual work area 20 and the target work area and / or the angle of the virtual work area 20 relative to the target work area. The measurement area adjustment button 24 is used to control the display or hiding of the measurement area adjustment pop-up window 23. If the measurement area adjustment pop-up window 23 is not displayed, the user clicks the measurement area adjustment button 24 to control the display of the measurement area adjustment pop-up window 23. If the measurement area adjustment pop-up window 23 is displayed, the user clicks the measurement area adjustment button 24 to control the hiding of the measurement area adjustment pop-up window 23.

[0201] For example, as shown in FIG16 , the full-screen overview image displays a virtual work area 20, a measurement area adjustment pop-up window 23, a measurement area adjustment button 24, and the target work area covered by the virtual work area 20. The user can use the measurement area adjustment pop-up window 23 to adjust the distance between the virtual work area 20 and the target work area and / or the angle of the virtual work area 20 relative to the target work area. The measurement area adjustment button 24 is used to control the display or hiding of the measurement area adjustment pop-up window 23. If the measurement area adjustment pop-up window 23 is not displayed, the user clicks the measurement area adjustment button 24 to control the display of the measurement area adjustment pop-up window 23. If the measurement area adjustment pop-up window 23 is displayed, the user clicks the measurement area adjustment button 24 to control the hiding of the measurement area adjustment pop-up window 23.

[0202] In some embodiments, the full-screen image can be a live view or an overview image, and the user can manually switch between the full-screen images. For example, when the full-screen image is a live view, the overview image is displayed via a preset window, with at least a portion of the preset window superimposed on the live view. In response to a user clicking on the preset window, the full-screen image switches from the live view to the overview image. When the full-screen image is an overview image, the live view is completely obscured. In response to a user clicking on the return button in the overview image, the full-screen image switches from the overview image to the live view. For example, as shown in FIG15 , after adjusting the virtual work area 20, the user clicks on the preset window 25, switching the full-screen image from the live view 10 to the overview image shown in FIG16 , where the virtual work area 20 in the overview image reflects the adjusted virtual work area. If the user clicks on the return button 26 , the control terminal switches the full-screen image from the overview image shown in FIG12 to the live view image.

[0203] In some embodiments, step S204 includes: adjusting the movable platform so that the viewing direction of the movable platform switches to another angle of the target work area, wherein the real-time viewing screen presents the target work area and the virtual work area at another angle, and the other angle is different from the viewing angle of the movable platform when the virtual work area is generated; adjusting the virtual work area according to the target work area and the virtual work area at the other angle. Specifically, adjusting the movable platform so that the viewing direction of the movable platform switches to the side of the target work area, and the real-time viewing screen includes the side of the target work area and the side of the virtual work area; when the viewing direction of the movable platform switches to the side of the target work area, performing a second operation to adjust the distance between the virtual work area and the target work area and / or adjust the angle of the virtual work area relative to the target work area, so that the adjusted virtual work area overlaps with the target work area or the deviation between the adjusted virtual work area and the target work area is less than a preset value. It should be noted that the first operation can also be performed to adjust the virtual work area from this side perspective.

[0204] For example, as shown in FIG17 , the live view screen 10 includes the side of the target work area 30 and the side of the virtual work area 20, as well as a measurement area adjustment pop-up window 23 and a measurement area adjustment button 24. Through the measurement area adjustment pop-up window 23, the user can adjust the distance between the virtual work area 20 and the target work area and / or the angle of the virtual work area 20 relative to the target work area. The measurement area adjustment button 24 is used to control the display or hiding of the measurement area adjustment pop-up window 23. When the measurement area adjustment pop-up window 23 is not displayed, the user clicks the measurement area adjustment button 24 to control the display of the measurement area adjustment pop-up window 23. When the measurement area adjustment pop-up window 23 is displayed, the user clicks the measurement area adjustment button 24 to control the hiding of the measurement area adjustment pop-up window 23. The live view screen 10 also displays a preset three-dimensional coordinate system 40. The plane formed by the X-axis and Y-axis of the preset three-dimensional coordinate system 40 is parallel to the virtual work area 20, and the Z-axis is perpendicular to the virtual work area 20. The preset three-dimensional coordinate system 40 changes synchronously with the adjustment of the virtual work area 20.

[0205] Step S205: planning an operating path of the movable platform based on the adjusted virtual operating area.

[0206] This embodiment facilitates the user to adjust the movable platform by displaying a real-time view screen with a pre-selection box superimposed thereon, so that the pre-selection box can at least partially cover the target work area, and the deviation between the virtual work area generated based on the pre-selection box and preset conditions and the target work area in the real-time view screen is small. This can reduce the time required for subsequent adjustment of the virtual work area, thereby reducing the path planning time as a whole and improving the efficiency of path planning.

[0207] In some embodiments, planning the operating path of the movable platform based on the adjusted virtual operating area may include generating the operating path of the movable platform based on the adjusted virtual operating area. The virtual operating area is determined to have been adjusted in response to a user confirming the virtual operating area. For example, as shown in FIG18 , upon detecting a user clicking a confirm button 27 , the virtual operating area 20 is determined to have been adjusted. At this point, the operating path of the movable platform may be generated based on the adjusted virtual operating area 20.

[0208] In some embodiments, planning the operating path of the movable platform based on the adjusted virtual operating area may include: generating an initial operating path based on the virtual operating area, and adaptively adjusting the initial operating path based on the adjustment to the virtual operating area, thereby obtaining an operating path planned based on the adjusted virtual operating area. It is understood that each time the virtual operating area is adjusted, the initial operating path is adaptively adjusted, and once the virtual operating area is adjusted, the initial operating path is also adjusted.

[0209] For example, as shown in FIG13 , the path planning interface displays a live view screen 10, a preselection box 11 displayed on the live view screen, and a measurement area confirmation icon 12. If a user clicks a physical shortcut key or the measurement area confirmation icon 12, the area currently selected by the preselection box 11 on the live view screen 10 is confirmed as the target work area. A virtual work area is generated on the live view screen based on the preselection box 11 and preset conditions. An initial work path is generated based on the virtual work area, and an overview map is also generated. As shown in FIG14 , the path planning interface includes the live view screen 10, a virtual work area 20 superimposed on the live view screen 10, a preset window 25, and an initial work path 41. After adjusting the virtual work area 20, the initial work path 41 is adaptively adjusted based on the adjustments to the virtual work area 20. The preset window 25 displays an overview map that includes the target work area, the virtual work area 20, and the initial work path 41. The user can drag the preset window 25 to any position on the path planning interface.

[0210] For example, as shown in Figure 15, the path planning interface also includes a measurement area adjustment button 24 and a preset window 25 for displaying an overview map. When a user clicks the measurement area adjustment button 24, the interface shown in Figure 15 switches to the overview map shown in Figure 16. Figure 16 also includes a measurement area adjustment pop-up window 23. The user adjusts the distance between the virtual operation area 20 and the target operation area through the distance adjustment control or distance input box in the measurement area adjustment pop-up window 23. The distance currently displayed in the distance input box is 3.2m. The distance adjustment control includes a first distance adjustment button, a second distance adjustment button, a third distance adjustment button and a fourth distance adjustment button. The first distance adjustment button is used to shorten the distance between the virtual operation area 20 and the target operation area by 1 meter, the second distance adjustment button is used to shorten the distance between the virtual operation area 20 and the target operation area by 0.1 meter, the third distance adjustment button is used to increase the distance between the virtual operation area 20 and the target operation area by 0.1 meter, and the fourth distance adjustment button is used to increase the distance between the virtual operation area 20 and the target operation area by 1 meter.

[0211] The user adjusts the angle of the virtual work area 20 relative to the target work area using the first angle adjustment control or the second angle adjustment control in the measurement area adjustment pop-up window 23. The first angle adjustment control is used to adjust the tilt angle of the virtual work area relative to the target work area around the X-axis, and the second angle adjustment control is used to adjust the tilt angle of the virtual work area relative to the target work area around the Y-axis. The first angle adjustment control includes a first button, a second button, a third button, and a fourth button. The first button is used to reduce the tilt angle of the virtual work area relative to the target work area around the X-axis by 1°, the second button is used to reduce the tilt angle of the virtual work area relative to the target work area around the X-axis by 0.1°, the third button is used to reduce the tilt angle of the virtual work area relative to the target work area around the X-axis by 0.1°, and the fourth button is used to reduce the tilt angle of the virtual work area relative to the target work area around the X-axis by 1°. The second angle adjustment control includes a fifth button, a sixth button, a seventh button and an eighth button. The fifth button is used to reduce the tilt angle of the virtual working area relative to the target working area around the Y-axis by 1°, the sixth button is used to reduce the tilt angle of the virtual working area relative to the target working area around the Y-axis by 0.1°, the seventh button is used to reduce the tilt angle of the virtual working area relative to the target working area around the Y-axis by 0.1°, and the eighth button is used to reduce the tilt angle of the virtual working area relative to the target working area around the Y-axis by 1°.

[0212] In the scenario shown in FIG15 , the user can adjust the viewing direction of the movable platform so that the real-time viewing image obtained by adjusting the viewing direction includes the side of the target work area and the side of the virtual work area, thereby obtaining the path planning interface shown in FIG17 . When a user clicks on the preset window 25 in FIG14 or FIG15 , the overview image in the preset window 25 is displayed full screen, and the path planning interface shown in FIG14 or FIG15 switches to the overview image shown in FIG16 . When a user clicks on the return button 26 in FIG16 , the full-screen overview image switches to the path planning interface shown in FIG14 or FIG15 .

[0213] In some embodiments, a job parameter adjustment operation is captured in a live view or overview image, and the initial job path or the job path is adjusted based on the job parameter adjustment operation. For example, the overview image shown in FIG19 displays a job parameter adjustment button 42 and a job parameter adjustment pop-up window 43. The user can adjust the initial job path 41 in the overview image through the job parameter adjustment pop-up window 43. The job parameter adjustment button 42 is used to control the display or hiding of the job parameter adjustment pop-up window 43. When the job parameter adjustment pop-up window 43 is not displayed and the user clicks the job parameter adjustment button 42, the job parameter adjustment pop-up window 43 is displayed. When the job parameter adjustment pop-up window 43 is displayed and the user clicks the job parameter adjustment button 42, the job parameter adjustment pop-up window 43 is hidden. The page shown in FIG19 also includes a save button 44. When the user clicks the save button 44, the latest job path is stored. When a user clicks the measurement area adjustment option in FIG19, the displayed page is switched to the page shown in FIG17. For another example, the path planning interface shown in FIG. 20 includes a real-time view screen 10 and an operation parameter adjustment pop-up window 43 , and the user can adjust the initial operation path 41 in the real-time view screen 10 through the operation parameter adjustment pop-up window 43 .

[0214] In some embodiments, the path planning method can be applied to a control terminal 200, which is configured to be in communication with a mobile platform 100. The control terminal 200 displays a real-time view of the mobile platform's view through a display device 210. In other embodiments, the path planning method can also be applied to the mobile platform 100. The mobile platform 100 can have a built-in display module for displaying the real-time view. The mobile platform can include an aerial photography aircraft, a ground-based viewing device, a mobile scanning device, a mobile measuring device, a mobile mapping device, a camera, or a mobile phone.

[0215] Please refer to Figure 21, which is a schematic flowchart of the steps of another path planning method provided in an embodiment of the present application.

[0216] As shown in FIG. 21 , the path planning method includes steps S301 to S303 .

[0217] Step S301: Determine the target operation area.

[0218] In this embodiment, the target operation area may be an area that requires surveying, aerial photography, spraying, sowing, placing, or cleaning. The target operation area may be a planar operation area, which may be a flat operation area, an inclined operation area, or a vertical operation area. For example, the target operation area may be farmland, the surface of a cliff, or one of the surfaces of a building.

[0219] In some embodiments, a target work area is determined in a displayed image, wherein the displayed image may include a real-time view or a map. A user's selection of the target work area is received in the displayed image, or the path planning device automatically determines the target work area, without requiring the movable platform to mark the target work area.

[0220] In some embodiments, determining the target work area includes at least one of: obtaining a user's selection of the target work area, or automatically determining the target work area based on preset target work area determination conditions. Obtaining the user's selection of the target work area includes at least one of: obtaining a user's operation on a physical shortcut key; or obtaining a user's click or circling operation on the screen. The user's operation on the physical shortcut key can be performed by clicking or sliding the physical shortcut key when the target work area appears on the live view screen, or when a viewfinder or pre-selected box frames the target work area, thereby determining the target work area. The preset target work area determination conditions can be set by the user or based on actual circumstances, and are not specifically limited in this embodiment of the present invention. For example, if the preset target work area determination condition is to identify a building with a height greater than a set height and a tower shape, then when the movable platform identifies a building with a height greater than the set height and a tower shape, it can determine one of the surfaces of the identified building as the target work area. This embodiment provides multiple methods for determining the target work area to meet different scenarios and improve the user experience.

[0221] In some embodiments, when a user clicks on the screen, one of the surfaces of the clicked object on the screen is identified as the target operating area, or an area of ​​a preset size centered on the clicked position on the screen is identified as the target operating area. The preset size can be set based on actual conditions, and the embodiments of the present application do not specifically limit this. For example, the preset size is 2cm*2cm or 3cm*3cm, etc. When a user circles the screen, the circled area is used as the target operating area.

[0222] In some embodiments, the displayed real-time view screen is displayed with a pre-selection box, which is used to indicate that the area framed on the real-time view screen is the target operation area. When the user's operation on the physical shortcut key is obtained, the area currently framed by the pre-selection box on the real-time view screen is confirmed as the target operation area. Alternatively, when the user's click operation on the measurement area confirmation icon on the screen is obtained, the area currently framed by the pre-selection box on the real-time view screen is confirmed as the target operation area, wherein the size of the pre-selection box on the real-time view screen is a preset size. For example, as shown in Figure 3, the displayed real-time view screen 10 is displayed with a pre-selection box 11 and a measurement area confirmation icon 12. At this time, if the user's click operation on the physical shortcut key or the measurement area confirmation icon 12 is obtained, the area currently framed by the pre-selection box 11 on the real-time view screen 10 is confirmed as the target operation area.

[0223] Step S302: Determine the distance between the target operation area and the movable platform, and generate a virtual operation area in the real-time view image of the movable platform based on parameters related to the distance.

[0224] In this embodiment, the virtual operation area is superimposed on the target operation area. In the real-time view image, the generated virtual operation area can be completely or partially superimposed on the target operation area, and the virtual operation area can also be located outside the target operation area.

[0225] In some embodiments, the operation of determining the distance between the target operation area and the movable platform can be real-time. It should be noted that the distance between the target operation area and the movable platform can refer to the distance between the target operation area and the movable platform body, or it can be the distance between the target operation area and the load on the movable platform body (such as an image sensor or a ranging sensor, etc.). In addition, it can be the distance between the representative point of the target operation area (such as the point of the target operation area corresponding to the center point of the pre-selected box) and the movable platform, or it can be the distance between multiple feature points on the target operation area and the movable platform. When the distance is the distance between the representative point of the target operation area (such as the point of the target operation area corresponding to the center point of the pre-selected box) and the movable platform, what is determined is the geographic location information of the point of the virtual operation area corresponding to the representative point, and based on the geometric information of the virtual operation area, the geographic location information of the entire virtual operation area can be determined.

[0226] In some embodiments, the planning method further includes: in response to the target working area being determined, automatically generating a virtual working area in a real-time view image of the movable platform based on parameters related to the distance.

[0227] In a further embodiment, in response to the target work area being determined, a virtual work area is immediately generated.

[0228] In a further embodiment, in response to the viewing direction of the movable platform facing the target working area and in response to the target working area being determined, a virtual working area is automatically generated in the real-time viewing screen of the movable platform based on parameters related to the distance.

[0229] For example, when the image sensor 130 is shooting at the target work area 30, the target work area 30 is displayed on the live view screen. For example, the pre-selection box 11 at least partially frames the target work area 30. In response to receiving the user's operation to determine the target work area 30, the virtual work area 20 is automatically generated on the live view screen. This embodiment can achieve the rapid generation of the virtual work area. It should be noted that the trigger condition for the generation of the virtual work area of ​​the present application can also be set to other conditions according to actual needs, for example, within a preset time after the target work area is determined or automatically generated upon receiving a preset operation.

[0230] In some embodiments, the distance-related parameters include: the distance between the target working area and the movable platform and / or the distance direction, wherein the distance between the target working area and the movable platform is determined along the distance direction, and the distance direction is the direction for determining the distance between the movable platform and the target working area.

[0231] In some embodiments, generating a virtual work area in the real-time view image of the movable platform based on the distance-related parameters may include: determining a distance between the virtual work area and the movable platform based on a distance between the target work area and the movable platform.

[0232] Specifically, the distance between the position corresponding to the virtual work area in real three-dimensional space and the movable platform is equal to or approximately equal to the distance between the target work area and the movable platform; or, there is a preset difference between the distance between the position corresponding to the virtual work area in real three-dimensional space and the movable platform and the distance between the target work area and the movable platform. The preset difference can be set according to actual needs.

[0233] In a further embodiment, based on the distance-related parameters, a virtual work area is generated in the real-time view screen of the movable platform, which also includes: determining the orientation of the virtual work area based on a fixed distance direction, so that the virtual work area is closer to the target work area in the circumferential direction surrounding the movable platform in the real three-dimensional space, wherein the fixed distance direction is the direction of measuring the distance between the movable platform and the target work area; or, determining the orientation of the virtual work area based on the viewing direction of the movable platform, generally the viewing direction is toward the target work area, if the virtual work area is projected according to the viewing direction, it can also be made closer to the target work area.

[0234] It should be noted that the above-mentioned preset condition one: determining the distance between the virtual work area and the movable platform based on the distance between the target work area and the movable platform, and preset condition two: determining the orientation of the virtual work area based on the fixed distance direction or determining the orientation of the virtual work area based on the framing direction of the movable platform, preset condition one and preset condition two may exist at the same time, or one of them may exist selectively. When the distance between the virtual work area and the movable platform is determined based on the distance between the target work area and the movable platform, the orientation of the virtual work area can be set according to actual needs. When the orientation of the virtual work area is determined based on the fixed distance direction or the framing direction of the movable platform, the distance between the virtual work area and the movable platform can be determined according to actual needs.

[0235] In some embodiments, the projection direction of the virtual work area is consistent with the fixed distance direction, or an angle is set between the projection direction of the virtual work area and the fixed distance direction; or the projection direction of the virtual work area is consistent with the framing direction, or an angle is set between the projection direction of the virtual work area and the framing direction.

[0236] When the orientation of the virtual work area is related to the fixed distance direction, the fixed distance direction is aligned with the projection direction of the virtual work area, making the virtual work area more closely aligned with the target work area. Alternatively, an angle is defined between the fixed distance direction and the projection direction of the virtual work area. The size of this angle can be adjusted based on actual conditions.

[0237] When the orientation of the virtual work area is related to the framing direction, the framing direction is aligned with the projection direction of the virtual work area, bringing the virtual work area closer to the target work area. Alternatively, an angle is defined between the framing direction and the projection direction of the virtual work area. The size of the angle can be adjusted based on actual conditions.

[0238] In some embodiments, the viewing direction of the movable platform is consistent with the distance direction, that is, the distance direction can be determined based on the viewing direction of the movable platform. In a specific embodiment, the viewing direction of the movable platform can be directed toward the target operation area, and the distance direction at this time is the viewing direction of the movable platform. It should be noted that the viewing direction of the movable platform and the distance direction can also be provided with an angle, that is, they are not in the same direction. Determining the distance direction by the viewing direction realizes the function of automatically determining the distance measurement point on the target operation area. For example, the viewing direction will determine the viewing direction of the pre-selected frame, changing the position indicated by the center of the pre-selected frame, that is, the position indicated by the center of the pre-selected frame can represent the viewing direction. When the viewing direction is adjusted to face the target operation area, the point in the actual target operation area corresponding to the center of the pre-selected frame can be used as the distance measurement point. At this time, the measured distance can be the distance between the center / center of gravity of the movable platform body or the center / center of gravity of the image sensor and the distance measurement point of the above-mentioned actual target operation area. The distance direction is the direction in which the center / center of gravity of the movable platform body or the center / center of gravity of the image sensor points to the distance measurement point of the actual target operating area.

[0239] It should be noted that when the movable platform is a panoramic viewfinder, since there is no fixed shooting direction, the fixed distance direction can be determined as the placement direction of the virtual operation area.

[0240] In some embodiments, the distance between the movable platform and the target operating area is measured by a ranging sensor; or the distance between the target operating area and the movable platform is determined based on a pre-generated perception map and the geographic location information of the movable platform itself. Specifically, in response to the determination of the target operating area, the distance between the target operating area and the movable platform can be determined based on the pre-generated perception map and the geographic location information of the movable platform itself. The perception map is a visual map pre-constructed based on the sensing data collected by the visual sensor of the mobile platform. The geographic location information of the target operating area can be obtained through the visual map, and then based on the geographic location information of the mobile platform itself and the geographic location information of the target operating area, the distance between the target operating area and the movable platform can be calculated. The pre-generated perception map and the geographic location information of the movable platform itself can accurately and quickly determine the distance between the target operating area and the movable platform.

[0241] In some embodiments, the viewing direction of the movable platform is perpendicular or approximately perpendicular to the target work area, so that the virtual work area generated on the live view image is parallel or approximately parallel to the target work area within the live view image. The viewing direction of the movable platform is the direction of the shooting optical axis of the image sensor on the movable platform. This embodiment can accurately determine the corresponding geographic location of the virtual work area in real three-dimensional space by using preset conditions and the geographic location information of the movable platform itself.

[0242] In some embodiments, the displayed live view image shows the distance between the target work area and the movable platform. For example, as shown in FIG3 , the live view image displays distance indication information 13 and first prompt information 14. Distance indication information 13 is used to indicate the distance between the target work area and the movable platform. In FIG3 , the distance between the target work area and the movable platform is 61.9 meters. First prompt information 14 prompts the user to determine the target work area. If the distance between the target work area and the movable platform exceeds a preset distance range, a second prompt information is output. For example, the distance between the target work area and the movable platform displayed in the live view image and / or the display color of the pre-selected box are changed to a preset color to indicate to the user that the distance measurement failed. The preset distance range can be set based on actual conditions and is not specifically limited in this embodiment of the present application. For example, if the preset distance range is [4, 150], the second prompt information is output if the distance between the target work area and the movable platform is less than 4 meters or greater than 150 meters.

[0243] In some embodiments, the path planning method provided in the embodiments of the present application, before step S201, further includes: obtaining a user's adjustment instruction, the adjustment instruction being used to adjust the viewing angle of the movable platform; adjusting the viewing angle of the movable platform according to the adjustment instruction so that the viewing direction of the movable platform is vertical or approximately vertical to the target operating area. Wherein, adjusting the viewing angle of the movable platform may include at least one of the following: adjusting the position of the movable platform, adjusting the posture of the movable platform, adjusting the posture of the gimbal of the movable platform, and the image sensor is connected to the gimbal. Wherein, the gimbal may be a three-axis gimbal. This embodiment adjusts the viewing angle of the movable platform based on the user's adjustment instruction so that the viewing direction of the movable platform is vertical or approximately vertical to the target operating area, thereby ensuring that the virtual operating area subsequently generated on the real-time view screen can be parallel or approximately parallel to the target operating area in the real-time view screen.

[0244] In some embodiments, the path planning method provided by the embodiment of the present application, before step S201, further includes: determining the angle of the target operating area based on a pre-generated perception map; automatically determining the viewing angle of the movable platform based on the angle of the target operating area, so that the viewing direction of the movable platform is perpendicular or approximately perpendicular to the target operating area. Wherein, the angle of the target operating area is the angle of the target operating area relative to the horizontal plane, and the angle of the target operating area can be obtained through the pre-generated perception map. This embodiment determines the angle of the target operating area based on the perception map, and based on the angle of the target operating area, it can automatically determine the viewing angle of the movable platform under the condition that the viewing direction is perpendicular or approximately perpendicular to the target operating area, thereby reducing user participation and ensuring that the virtual operating area subsequently generated on the real-time view screen can be parallel or approximately parallel to the target operating area in the real-time view screen.

[0245] In some embodiments, generating a virtual work area on the live view screen of the mobile platform based on parameters related to distance includes: generating a virtual work area on the live view screen based on the parameters related to distance and a pre-selection box. In a further embodiment, in response to the viewing direction of the mobile platform facing the target work area, and in response to the target work area being determined, generating a virtual work area on the live view screen based on the parameters related to distance and the pre-selection box. For example, as shown in Figures 3 and 4, the live view screen 10 shown in Figure 3 displays a pre-selection box 11 and a measurement area confirmation icon 12. At this time, if a user clicks on the physical shortcut key or the measurement area confirmation icon 12, the area currently selected by the pre-selection box 11 on the live view screen 10 is confirmed as the target work area. At this time, the virtual work area 20 in Figure 3 is generated on the live view screen 10 based on the pre-selection box 11 and the parameters related to distance.

[0246] The size of the virtual work area in the live view is determined based on a preselected box, and the position of the virtual work area is determined based on parameters related to the distance of the target work area from the movable platform. Specifically, the position of the virtual work area in real three-dimensional space is determined based on the distance of the target work area from the movable platform. This embodiment determines the size of the virtual work area in the live view based on the preselected box, thereby simplifying the determination of the virtual work area. During the actual planning process, it is only necessary to adjust the distance between the movable platform and the target work area so that the preselected box at least partially covers the target work area. The corresponding size of the preselected box in real three-dimensional space is determined based on the measured distance. In other words, the larger the distance value, the larger the size of the preselected box in real three-dimensional space. Conversely, the smaller the distance value, the smaller the size of the preselected box in real three-dimensional space. By simply adjusting the movable platform so that the preselected box covers the target work area as much as possible, the generated virtual work area can be made to fit the target work area as closely as possible.

[0247] In some embodiments, the size of the pre-selection box in the real-time view screen is a preset size. The size of the pre-selection box can be set in advance according to actual needs. Combined with the release distance of the generated virtual work area and the preset size of the pre-selection box, the geographical location of the virtual work area can be determined more easily.

[0248] In some embodiments, the path planning method provided by the embodiments of the present application, before step S301, further includes: displaying a pre-selected box on the real-time view screen; and obtaining a framing adjustment operation, and adjusting the framing direction and size according to the framing adjustment operation so that the pre-selected box at least partially covers the target working area. The framing adjustment operation includes at least one of the following: a position adjustment operation of the movable platform body; a posture adjustment operation of the movable platform body; a posture adjustment operation of the image sensor carried by the movable platform; and a focal length adjustment operation of the image sensor carried by the movable platform. The position adjustment operation of the movable platform body is used to adjust the position of the movable platform, the posture adjustment operation of the movable platform body is used to adjust the posture of the movable platform, the posture adjustment operation of the image sensor carried by the movable platform body is used to adjust the posture of the image sensor, and the focal length adjustment operation of the image sensor carried by the movable platform body is used to adjust the focal length of the image sensor. This embodiment adjusts the movable platform so that the pre-selection box can at least partially cover the target work area, and the deviation between the virtual work area generated according to the distance from the target work area to the movable platform and the pre-selection box and the target work area in the real-time view screen is small. This can reduce the time required for subsequent adjustment of the virtual work area, thereby reducing the path planning time as a whole and improving the efficiency of path planning.

[0249] In some embodiments, generating a virtual work area in the live view of the mobile platform based on the distance-related parameters may include: extracting a contour line of the target work area in the live view based on the determined target work area; and generating the virtual work area in the live view based on the area delineated by the contour line and the determined distance-related parameters. Specifically, the size of the virtual work area in the live view is determined based on the area delineated by the contour line, and the position of the virtual work area in real three-dimensional space is determined based on the determined distance-related parameters. In further embodiments, the virtual work area may be generated in the live view based on the area delineated by the contour line and the determined distance-related parameters in response to the viewing direction of the mobile platform facing the target work area and in response to the target work area being determined. This embodiment extracts the contour line of the target work area in the live view, and generates a virtual work area in the live view that covers the target work area based on the area delineated by the contour line in the live view and preset conditions. This reduces the deviation between the virtual work area and the target work area, reduces the time required for subsequent adjustment of the virtual work area, and further improves overall path planning efficiency.

[0250] In some embodiments, determining the distance between the target work area and the movable platform, and generating a virtual work area in the real-time view of the movable platform based on the parameters related to the distance includes: determining multiple feature points based on the determined target work area; obtaining the distance between each feature point and the movable platform; determining the geographic location information of each feature point based on the geographic location information of the movable platform and the parameters related to the distance; and generating the virtual work area based on the geographic location information of each feature point. In a further embodiment, in response to the viewing direction of the movable platform facing the target work area and in response to the target work area being determined, a virtual work area covering the target work area is generated and displayed based on the geographic location information of each feature point. In this embodiment, the geographic location information of the multiple feature points can be determined based on the relative positional relationship between the multiple feature points in the target work area and the movable platform, as well as the geographic location information of the movable platform. Based on the geographic location information of the feature points, the positions of the feature points can be determined, thereby generating and displaying a virtual work area covering the target work area.

[0251] Furthermore, based on the geographic location information of each feature point, the step of generating a virtual operation area includes: superimposing a point marker on the corresponding feature point according to the geographic location information of the feature point; specifically, based on the geographic location information of the feature point, the position of the feature point in the real-time view image can be determined, so that the point marker is superimposed on the corresponding position, so as to realize superimposing the point marker on the corresponding feature point. Among them, the specific display style of the point marker can be set based on the actual situation, and the embodiment of the present application does not make specific restrictions on this. For example, the point marker is a white dot, a red dot or a black dot of a preset pixel size. By connecting multiple point markers according to the actual relative position relationship between multiple feature points, a virtual operation area covering the target operation area can be obtained, which reduces the deviation between the virtual operation area and the target operation area, reduces the time required for subsequent adjustment of the virtual operation area, and further improves the overall path planning efficiency.

[0252] In some embodiments, the step of determining multiple feature points includes: obtaining an operation of the user to determine multiple feature points, such as the user clicking on the real-time view screen; or, automatically extracting multiple feature points. In this embodiment, a preset feature point extraction algorithm can be used to automatically extract feature points to extract multiple feature points of the target operating area in the real-time view screen. Among them, the preset feature point extraction algorithm can be set based on actual conditions, and the embodiment of the present application does not make specific limitations on this. For example, the preset feature point extraction algorithm includes at least one of the following: SIFT (Scale-Invariant Feature Transform) algorithm, SURF (Speeded-Up Robust Feature) algorithm, ORB (Oriented FAST and Rotated BRIEF) algorithm, Haris corner detection algorithm, and Hessian-Laplace algorithm.

[0253] In some embodiments, the multiple feature points determined include at least the endpoints of the boundary line of the target work area. For example, as shown in FIG6 , the target work area determined is the surface area 15 of a building. Therefore, multiple feature points of the building surface area 15 can be extracted in the live view image 10, and the extracted multiple feature points include the endpoints of the boundary line of the building surface area 15. Because the multiple feature points extracted include at least the endpoints of the boundary line of the target work area, the virtual work area subsequently generated based on the multiple feature points can effectively cover the target work area in the live view image.

[0254] In some embodiments, the distance-related parameters include the distance between the feature point and the movable platform, and the measurement direction when measuring the distance between the feature point and the movable platform. In the aforementioned description of the distance-related parameters, the distance-related parameters include: the distance between the target work area and the movable platform. In this embodiment, it represents the distance between each feature point on the target work area and the movable platform. Correspondingly, what is determined is the distance between each corresponding point of the virtual work area and the movable platform, and the orientation of each corresponding point of the virtual work area is also determined based on the fixed distance direction of each feature point.

[0255] Step S303: planning an operation path based on the virtual operation area.

[0256] In some embodiments, the planned operation path includes a framing path, an inspection path, a spreading path, a spraying path, a cleaning path, etc.

[0257] In some embodiments, based on the virtual work area, planning the work path may include: automatically generating a "bow" or "Z" shaped work path based on the virtual work area, the work parameters set by the user, and the preset planning rules. Among them, the preset planning rules can be set based on actual conditions, and the embodiments of the present application do not specifically limit this. The work parameters set by the user include surveying and mapping parameters, aerial photography parameters, spraying parameters, sowing parameters or scanning parameters, etc. For example, the movable platform is an aerial photography aircraft, and the work parameters set by the user include the ground sampling distance, the distance of the path relative to the target work area, the route speed, the route direction, the altitude mode, the safe take-off height, the heading overlap rate, the lateral overlap rate and the route starting point, etc. The automatic work path of this embodiment based on the virtual work area, the work parameters set by the user, and the preset planning rules is more in line with the user's needs, and can ensure the matching degree between the work path and the target work area, thereby ensuring the work effect when the movable platform uses the work path to perform operations, and the user experience is better.

[0258] In some embodiments, the path planning method provided by embodiments of the present application further includes: adjusting the virtual work area according to the target work area; and planning the work path based on the adjusted virtual work area. The adjustment is performed so that the adjusted virtual work area coincides with the target work area or the deviation between the adjusted virtual work area and the target work area is less than a preset value. In other embodiments, the virtual work area can also be adjusted based on the user's actual needs.

[0259] In some scenarios, it is impossible to accurately know the relative position relationship between the target operation area and the virtual operation area, such as errors in the measured distance, which makes it impossible to directly obtain the geographic location information of the target operation area. Therefore, through the pre-generated virtual operation area, the geographic location information of the virtual operation area can be accurately known, and through adjustment operations, the change in geographic location information corresponding to the adjustment operation can also be determined according to preset rules. Therefore, through the virtual operation area and adjustment operations, when the user visually determines in the real-time view screen that the virtual operation area coincides with the target operation area or the deviation between the virtual operation area and the target operation area is less than the preset value, the geographic location information of the target operation area can be inferred.

[0260] In some embodiments, when the virtual operating area is adjusted to coincide with the target operating area, the geographic location information of the adjusted virtual operating area can substantially represent the geographic location information of the target operating area. Therefore, planning an operating path based on the adjusted virtual operating area information is equivalent to planning an operating path based on the acquired geographic location information of the target operating area. In other embodiments, the virtual operating area can be adjusted based on actual needs, and the adjusted virtual operating area may not coincide with the target operating area.

[0261] In some embodiments, during the step of adjusting the virtual operating area according to the target operating area, a user may visually determine whether the adjusted virtual operating area and the target operating area overlap or whether the deviation between the virtual operating area and the target operating area is less than a preset value, or the user may automatically determine whether the adjusted virtual operating area and the target operating area overlap or whether the deviation between the virtual operating area and the target operating area is less than a preset value, which is not specifically limited in the embodiments of the present application. The overlap between the adjusted virtual operating area and the target operating area means that the adjusted virtual operating area fits the target operating area perfectly, and the outlines of the adjusted virtual operating area and the target operating area overlap.

[0262] In some embodiments, adjusting the virtual operating area according to the target operating area includes: adjusting the virtual operating area based on a user's area adjustment operation obtained, wherein the user adjusts the virtual operating area according to the target operating area; or automatically adjusting the virtual operating area according to the target operating area and a preset area adjustment algorithm. The following embodiments will be described using a user's manual area adjustment operation to adjust the virtual operating area as an example. However, those skilled in the art will appreciate that most of the following embodiments are also applicable to automatic adjustment methods.

[0263] In some embodiments, the step of adjusting the virtual operating area according to the target operating area may include at least a first operation and / or a second operation, wherein the first operation is used to adjust at least one of the size, shape, position, and orientation of the virtual operating area, and the second operation is used to adjust the distance between the virtual operating area and the target operating area and / or the angle of the virtual operating area relative to the target operating area. In this embodiment, by adjusting at least one of the size, shape, position, and orientation of the virtual operating area, the outline of the adjusted virtual operating area and the outline of the target operating area are made as close as possible; by adjusting the distance between the virtual operating area and the target operating area and / or the angle of the virtual operating area relative to the target operating area, the adjusted virtual operating area and the target operating area can be made to fit as closely as possible. In general, by at least one of the first operation and the second operation, the adjusted virtual operating area and the target operating area are overlapped or the deviation between the adjusted virtual operating area and the target operating area is less than a preset value.

[0264] In some embodiments, the deviation between the virtual operation area and the target operation area is less than a preset value, including at least one of the following: the distance between the virtual operation area and the target operation area in the direction of the Z axis of the preset three-dimensional coordinate system is less than a first preset value, the plane formed by the X axis and the Y axis of the preset three-dimensional coordinate system is parallel to the virtual operation area, and the Z axis is perpendicular to the virtual operation area; the inclination angle between the virtual operation area and the target operation area is less than a second preset value, and the inclination angle is the angle at which the virtual operation area is tilted relative to the target operation area around the X axis or the Y axis; the distance between the outline of the virtual operation area and the outline of the target operation area is less than a third preset value; the deviation angle between the virtual operation area and the target operation area in the direction around the Z axis is less than a fourth preset value.

[0265] In some embodiments, the path planning method provided by embodiments of the present application further includes: displaying a preset three-dimensional coordinate system on the virtual work area. The preset three-dimensional coordinate system also adaptively adjusts in response to area adjustment operations. By displaying the preset three-dimensional coordinate system on the virtual work area, this embodiment facilitates the user to determine the deviation between the virtual work area and the target work area using the preset three-dimensional coordinate system, providing a reference for the user to adjust the virtual work area, thereby improving the convenience and accuracy of adjusting the virtual work area.

[0266] In some embodiments, the step of adjusting the virtual operation area according to the target operation area includes: adjusting the movable platform so that the viewing direction of the movable platform switches to another angle of the target operation area, and the real-time viewing screen presents the target operation area and the virtual operation area at the other angle, the other angle being different from the viewing angle of the movable platform when the virtual operation area is generated; and adjusting the virtual operation area according to the target operation area and the virtual operation area at the other angle. In this embodiment, the target operation area and the virtual operation area at the other angle are used to supplement different perspectives to determine whether the virtual operation area and the target operation area overlap or whether the deviation between the virtual operation area and the target operation area is less than a preset value. Then, if it is determined that the virtual operation area and the target operation area do not overlap or the deviation between the virtual operation area and the target operation area is greater than or equal to the preset value, the virtual operation area is automatically or manually adjusted by the user to obtain an adjusted virtual operation area, thereby improving the accuracy and convenience of adjusting the virtual operation area.

[0267] In this embodiment, the virtual operating area can be adjusted by obtaining the target operating area and the virtual operating area at the angle when the virtual operating area is obtained. The adjustment operation may include the above-mentioned first operation and second operation. At the same time, the virtual operating area can be adjusted by changing the viewing angle so that the real-time viewing screen presents the target operating area and the virtual operating area at different angles. The adjustment operation may also include the above-mentioned first operation and second operation.

[0268] In some embodiments, the target operating area is a planar area, and the step of adjusting the virtual operating area according to the target operating area includes: adjusting the movable platform so that the viewing direction of the movable platform switches to the side of the target operating area, and the real-time viewing image includes the side of the target operating area and the side of the virtual operating area; when the viewing direction of the movable platform switches to the side of the target operating area, performing a second operation to adjust the distance between the virtual operating area and the target operating area and / or adjust the angle of the virtual operating area relative to the target operating area, so that the adjusted virtual operating area overlaps with the target operating area or the deviation between the adjusted virtual operating area and the target operating area is less than a preset value. It should be noted that the first operation can also be performed to adjust the virtual operating area under this side perspective.

[0269] In some embodiments, the path planning method provided by the embodiments of the present application, before step S301, further includes: adjusting the movable platform so that the viewing direction of the movable platform is the front of the target operation area, and the real-time view image contains at least a portion of the front of the target operation area. Among them, the position, posture of the movable platform body and / or the position or posture of the image sensor can be adjusted automatically or manually by the user to change the viewing direction of the movable platform so that the viewing direction of the movable platform is switched to the front of the target operation area. Since the front of the target operation area can fully display the size, shape, position and orientation of the target operation area, therefore, when the front of the target operation area is included in the real-time view image and a virtual operation area is superimposed on the real-time view image, at least one of the size, shape, position and orientation of the virtual operation area can be adjusted so that the adjusted virtual operation area can cover the target operation area.

[0270] In some embodiments, adjusting the virtual work area according to the target work area includes: adjusting at least one of the size, shape, position and orientation of the virtual work area according to the target work area on the front. Wherein, when the viewing direction of the movable platform is the front of the target work area, a first operation can be performed to adjust at least one of the size, shape, position and orientation of the virtual work area. Since the front of the target work area can fully display the size, shape, position and orientation of the target work area, when the front of the target work area is included in the real-time view screen and the virtual work area is superimposed on the real-time view screen, at least one of the size, shape, position and orientation of the virtual work area can be adjusted so that the adjusted virtual work area can cover the target work area. It should be noted that, under this frontal perspective, the second operation can also be performed to adjust the virtual work area.

[0271] In some embodiments, adjusting the virtual operation area according to the target operation area includes adjusting the virtual operation area based on an acquired user area adjustment operation, wherein the user adjusts the virtual operation area according to the target operation area. For example, the virtual operation area includes at least one adjustment point located on a boundary of the virtual operation area, and the area adjustment operation includes a user dragging or clicking the adjustment point. The user dragging or clicking the adjustment point is used to adjust the size and / or shape of the virtual operation area. In other embodiments, the virtual operation area includes multiple endpoint add buttons and / or endpoint delete buttons, and the area adjustment operation includes a user dragging or clicking the endpoint add button and / or endpoint delete button. The user dragging or clicking the endpoint add button is used to add an endpoint to the virtual operation area to adjust the size and / or shape of the virtual operation area, and the user dragging or clicking the endpoint delete button is used to delete an endpoint to adjust the size and / or shape of the virtual operation area. In other embodiments, at least one endpoint add button and / or endpoint delete button is displayed on each boundary of the virtual operation area.

[0272] In some embodiments, the step of adjusting the virtual operating area according to the target operating area may also include: automatically adjusting the virtual operating area according to the target operating area and a preset area adjustment algorithm, so that the adjusted virtual operating area coincides with the target operating area or the deviation between the adjusted virtual operating area and the target operating area is less than a preset value. Wherein, the preset area adjustment algorithm can be set based on actual conditions, and the embodiments of the present application do not make specific limitations on this. For example, the deviation between the target operating area and the virtual operating area is automatically determined, and according to the deviation between the target operating area and the virtual operating area, the size, shape, position, orientation of the virtual operating area, the distance between the virtual operating area and the target operating area, and / or the angle of the virtual operating area relative to the target operating area are automatically adjusted, so that the adjusted virtual operating area coincides with the target operating area or the deviation between the adjusted virtual operating area and the target operating area is less than a preset value.

[0273] In some embodiments, the step of acquiring a user's area adjustment operation includes at least one of the following: acquiring the area adjustment operation in a live view image; acquiring the area adjustment operation in an overview image, wherein the overview image displays the target work area and the virtual work area framed at a preset angle by the movable platform. The overview image is a captured image. This embodiment enables adjustment of the virtual work area based on the live view image and / or the overview image, allowing the user to use the appropriate adjustment method according to actual needs, thereby improving the user experience.

[0274] In some embodiments, when a region adjustment operation is performed on the live view screen, the virtual work area in the overview map also changes in response to the region adjustment operation; and / or when a region adjustment operation is performed on the overview map, the virtual work area in the live view screen also changes in response to the region adjustment operation. This embodiment enables synchronous adjustment of the virtual work area in the live view screen and the virtual work area in the overview map, ensuring consistency between the virtual work area in the live view screen and the virtual work area in the overview map.

[0275] In some embodiments, the path planning method provided by the embodiments of the present application further includes: automatically generating an overview map in response to the generation of the virtual work area. This embodiment automatically generates the overview map at the same time as the virtual work area is generated, thereby enabling the overview map to be generated quickly and improving the user experience.

[0276] In some embodiments, the route planning method provided in embodiments of the present application further includes: obtaining an overview map in response to a user's photo operation. In this embodiment, the overview map obtained in response to the user's photo operation better meets the user's needs, avoiding situations where an automatically generated overview map fails to meet the user's needs. The overview map can be regenerated by taking a photo, thereby improving the user experience.

[0277] In some embodiments, the path planning method provided in embodiments of the present application further includes: automatically storing the overview map after obtaining it, or storing the overview map in response to a user's request to store the overview map. By storing the overview map, this embodiment enables the virtual work area to be adjusted using the stored overview map even if the communication connection with the mobile platform is disconnected, thereby improving the user experience.

[0278] In some embodiments, a perception map is superimposed on the live view display. The perception map includes a perception model of the target work area, which is used to provide a reference basis for area adjustment operations. In this embodiment, by superimposing the perception map on the live view display of the mobile platform, and the perception map includes a perception model of the target work area, the displayed perception model can be used to determine how to adjust the virtual work area so that the adjusted virtual work area coincides with the target work area or the deviation between the adjusted virtual work area and the target work area is less than a preset value, thereby improving the efficiency and accuracy of virtual work area adjustment.

[0279] In some embodiments, in the step of adjusting the virtual operation area according to the target operation area: the adjusted geographic location information of the virtual operation area can be determined based on the obtained geographic location information of the virtual operation area and the corresponding change in geographic location information when the virtual operation area is adjusted.

[0280] In some embodiments, the geographical location information of the virtual operation area can be determined based on the geographical location information of the movable platform and the above-mentioned preset conditions. The geographical location information of the movable platform can be determined based on its own positioning module.

[0281] In some embodiments, the amount of change in geographic location information corresponding to adjustments to the virtual work area can be determined based on preset rules and automatically acquired. For example, when adjusting and dragging the boundaries of the virtual work area on the display interface, the distance moved on the display interface can be calculated as the amount of change in real three-dimensional space based on a preset mapping algorithm. When the virtual work area is moved laterally, the distance moved on the display interface can be calculated as the amount of change in real three-dimensional space based on a preset mapping algorithm. Similarly, the amount of change in geographic location information associated with adjustments to the virtual work area on the display interface can be automatically acquired.

[0282] In some embodiments, planning the work path based on the adjusted virtual work area may include generating a work path based on the adjusted virtual work area. The adjustment of the virtual work area is determined to be complete in response to a user confirming the virtual work area. This embodiment generates a work path for the movable platform based on the adjusted virtual work area after the adjustment of the virtual work area, thereby ensuring a good match between the work path and the target work area, thereby ensuring a satisfactory work effect when the movable platform uses the work path.

[0283] In some embodiments, planning an operation path based on the adjusted virtual operation area may include: generating an initial operation path based on the virtual operation area, and adaptively adjusting the initial operation path based on the adjustment to the virtual operation area, thereby obtaining an operation path planned based on the adjusted virtual operation area. It is understandable that each time the virtual operation area is adjusted, the initial operation path is adaptively adjusted, and after the virtual operation area has been adjusted, the initial operation path is also adjusted. This embodiment generates an initial operation path based on the generated virtual operation area while generating the virtual operation area, and then adaptively adjusts the initial operation path while adjusting the virtual operation area. In this way, after the virtual operation area has been adjusted, the initial operation path is also adjusted, thereby obtaining a final operation path, thereby improving the real-time performance of path planning.

[0284] In some embodiments, based on the adjusted virtual work area, planning the work path may include: automatically generating a "bow" or "Z" shaped work path based on the virtual work area, the work parameters set by the user, and the preset planning rules. Among them, the preset planning rules can be set based on actual conditions, and the embodiments of the present application do not specifically limit this. The work parameters set by the user include surveying and mapping parameters, aerial photography parameters, spraying parameters, sowing parameters or scanning parameters, etc. For example, the movable platform is an aerial photography aircraft, and the work parameters set by the user include the ground sampling distance, the distance of the path relative to the target work area, the route speed, the route direction, the altitude mode, the safe take-off height, the heading overlap rate, the lateral overlap rate and the route starting point, etc. The automatic work path of this embodiment based on the adjusted virtual work area, the work parameters set by the user, and the preset planning rules is more in line with the user's needs, and can ensure the matching degree between the work path and the target work area, thereby ensuring the work effect when the movable platform uses the work path to perform operations, and the user experience is better.

[0285] In some embodiments, after generating a work path, the path planning method provided in embodiments of the present application further includes: determining whether there is a path segment in the work path that collides with an obstacle; and if so, generating a prompt message. The prompt message may include marking the path segment in the work path that collides with an obstacle. By marking the path segment in the work path that collides with an obstacle, this embodiment allows the user to be aware of the collision risk of the path segment, making it easier for the user to adjust the path.

[0286] Furthermore, it is possible to determine whether there is a path segment in the operation path that collides with an obstacle based on the pre-generated perception map.

[0287] In some embodiments, the path planning method provided by embodiments of the present application further includes: obtaining a user's operation parameter adjustment operation in a real-time view image and / or obtaining a user's operation parameter adjustment operation in an overview image; and adaptively adjusting the operation path based on the operation parameter adjustment operation. This embodiment can adjust operation parameters based on the real-time view image and / or the overview image, allowing the user to select an appropriate adjustment method to adjust operation parameters according to actual needs, thereby improving the user experience.

[0288] In some embodiments, after generating the work path, the path planning method provided in embodiments of the present application further includes: adjusting the work path based on a pre-generated perception map, and displaying the adjusted work path, so that the adjusted work path does not have any path segments that collide with obstacles. Alternatively, in response to a user's adjustment operation on the work path, adjusting the work path and displaying the adjusted work path, so that the adjusted work path does not have any path segments that collide with obstacles.

[0289] In some embodiments, the movable platform can operate according to the operation path, that is, the planning device and the device that performs the operation can be the same device, thereby realizing the integration of planning operations; or, the generated operation path can be sent to another movable platform, so that the operation path can be planned in advance, and other operation equipment does not need to plan the operation path again.

[0290] In some embodiments, the path planning method can be applied to a control terminal 200, which is configured to be in communication with a mobile platform 100. The control terminal 200 displays a real-time view of the mobile platform's view through a display device 210. In other embodiments, the path planning method can also be applied to the mobile platform 100. The mobile platform 100 can have a built-in display module for displaying the real-time view. The mobile platform can include an aerial photography aircraft, a ground-based viewing device, a mobile scanning device, a mobile measuring device, a mobile mapping device, a camera, or a mobile phone.

[0291] Please refer to Figure 22, which is a schematic flowchart of the steps of another path planning method provided in an embodiment of the present application.

[0292] As shown in FIG. 22 , the path planning method includes steps S401 to S404 .

[0293] Step S401: Display the three-dimensional shape of the target object in the image.

[0294] In this embodiment, the image is a 2D image or a 3D image acquired by an image sensor carried by a movable platform; or it is a 2D image or a 3D image generated based on the sensing information of a ranging sensor carried by a movable platform. The image sensor and / or the ranging sensor are carried on a movable platform. The target object can be any object in the image, or any object that meets the set significance conditions, or an object selected by a pre-selection box in the image, or any object selected by the user, or any object that meets the requirements set by the user. The embodiments of the present application do not specifically limit this. The three-dimensional shape of the target object refers to the three-dimensional figure formed by the entire target object. The three-dimensional shape can truly reflect the appearance of the target object in real three-dimensional space, including not only the outline of the target object, but also the texture undulations within the outline.

[0295] In some embodiments, the image is a model image or a real-time view image.

[0296] In some embodiments, displaying the 3D shape of the target object in the image may include: displaying the 3D shape of the target object based on a pre-generated 3D model; or displaying the 3D shape of the target object based on a real-time view. The 3D model may be a mesh 3D model or a point cloud 3D model.

[0297] Step S402: In response to the target surface determination operation, determine the target surface of the target object from the three-dimensional shape of the target object.

[0298] In this embodiment, the target surface determination operation includes a user's long press or click operation on the target surface preselection mark, a user's click operation on the three-dimensional shape, and / or a user's click operation on a physical shortcut key, etc., which are not limited here. A target surface preselection mark is displayed in the image, and the target surface preselection mark is used to indicate that the surface area marked is the target surface. The user can drag the target surface preselection mark to change the position of the target surface preselection mark, or change the position of the target surface preselection mark through a physical key. In response to the user's long press or click operation on the target surface preselection mark, the surface area marked on the three-dimensional shape of the target object by the target surface preselection mark is determined as the target surface, or in response to the user's click operation on the three-dimensional shape, the surface area marked on the three-dimensional shape of the target object by the target surface preselection mark is determined as the target surface.

[0299] In some embodiments, the target surface and the target working area in the aforementioned embodiments may have the same meaning and may refer to the same object or area.

[0300] In some embodiments, the three-dimensional shape of the target object includes a surface of the target object, and the target surface is related to the surface of the target object. For example, the target surface may be one of the surfaces of the target object. In response to the target surface determination operation, determining the target surface of the target object from the three-dimensional shape of the target object may include: in response to a user clicking operation on the three-dimensional shape, determining the clicked surface of the three-dimensional shape as the target surface of the target object. Alternatively, in response to a user selecting operation on one of the surfaces of the three-dimensional shape, selecting the corresponding surface of the three-dimensional shape of the target object as the target surface.

[0301] The target object may be an object that needs to be surveyed, photographed, sprayed, spread, placed, or cleaned, such as a building, a mountain, a river, or a farmland. The target surface and the target operation area in the above embodiment may refer to the same area.

[0302] Step S403: Automatically generate a reference surface based on the target surface.

[0303] In this embodiment, the deviation between the reference surface generated based on the target surface and the target surface is less than a preset value, wherein the preset value can be set based on actual conditions and is not specifically limited in this embodiment of the application.

[0304] In some embodiments, the reference surface and the virtual working area in the aforementioned embodiments may have the same meaning and may refer to the same object or area.

[0305] In some embodiments, at least a portion of the reference surface is substantially parallel to the target surface, or the reference surface is superimposed on the target surface of the target object. In this embodiment, since at least a portion of the reference surface is substantially parallel to the target surface, or the reference surface is superimposed on the target surface of the target object, it is convenient for the user to observe whether the deviation between the reference surface and the target surface is less than a preset value or whether the reference surface and the target surface overlap.

[0306] In some embodiments, the displayed image is a real image, and the reference surface is a virtual image superimposed on the real image. Alternatively, the image displayed on the control terminal and the reference surface are virtual images. For example, the control terminal displays a three-dimensional model, and the reference surface is a related virtual image generated based on the three-dimensional model.

[0307] Further, in response to generating the reference surface, the reference surface is displayed.

[0308] In some embodiments, automatically generating a reference surface based on a target surface may include: determining the geographic location information of the reference surface based on the geographic location information of the movable platform and the relative positional relationship between the target surface and the movable platform; and automatically generating the reference surface based on the geographic location information of the reference surface. The image is a real-time view of the movable platform. Specifically, the relative positional relationship between the target surface and the movable platform is determined using information acquired by a ranging sensor. The relative positional relationship may include the distance of the target surface relative to the movable platform and / or the distance direction when measuring the distance. By acquiring the distance and distance direction of the target surface relative to the movable platform, the corresponding position of the reference surface in real three-dimensional space is determined. It should be noted that the distance between the target surface and the movable platform may refer to the distance between the target surface and the movable platform itself, or the distance between the target surface and a load on the movable platform itself (such as an image sensor or ranging sensor). The distance between the target surface and the movable platform may be the distance between a representative point on the target surface (such as a point in the target operating area corresponding to the center point of a pre-selected box) and the movable platform, or the distance between multiple feature points on the target surface and the movable platform. When the distance is the distance between a representative point of the target surface (e.g., a point in the target operating area corresponding to the center point of the preselected box) and the movable platform, the geographic location information of the point on the reference surface corresponding to the representative point is determined. Based on the geometric information of the reference surface, the geographic location information of the entire reference surface can be determined. The geometric information of the reference surface can be preset, such as the preselected box in the aforementioned embodiment.

[0309] The reference surface generated by obtaining the distance of the target surface relative to the movable platform through the ranging sensor can fit the target surface well or the deviation between the reference surface and the target surface is less than the preset value, so that the reference surface fits the target surface as much as possible, and the operation path can be planned based on the reference surface.

[0310] In some embodiments, automatically generating a reference surface based on a target surface may include: determining geographic location information of the target surface based on a pre-generated three-mode model; and automatically generating the reference surface based on the geographic location information of the target surface. Specifically, the pre-generated three-mode model can accurately obtain the geographic location information of each target object, thereby simplifying the generation of the reference surface.

[0311] In some embodiments, the generated reference surface at least partially covers or fits the target surface, so that the reference surface more accurately restores the target surface to improve operation accuracy.

[0312] In some embodiments, the path planning method provided in embodiments of the present application further includes: changing the displayed image, where the changed image has a different viewing angle than the image before the change; and continuing to display the reference surface in the changed image, where the changed image includes the target surface and the reference surface at the different viewing angles. This embodiment, by changing the image so that the changed image includes the target surface and the reference surface at the different viewing angles, facilitates the user's visual inspection of the deviation between the reference surface and the target surface at the different viewing angles, thereby improving the ease and accuracy of adjusting the reference surface.

[0313] Step S404: Generate an operation path of the movable platform based on the reference surface.

[0314] The entire implementation process of the embodiment of the present application does not require determining the target operating area based on the marking of the movable platform, which reduces the complexity of path planning and improves the path planning efficiency of the movable platform. In addition, since the reference surface displayed in the image is generated based on the target surface of the target object, the operating path of the movable platform is generated based on the reference surface, which can ensure the matching degree between the operating path and the target surface, thereby ensuring the operating effect of the movable platform when using the operating path to perform operations, and providing a better user experience.

[0315] In some embodiments, at least a portion of the working path is generated on a reference surface, or at least a portion of the working path is substantially parallel to the reference surface. In this embodiment, since at least a portion of the working path is generated on a reference surface, or at least a portion of the working path is substantially parallel to the reference surface, the working effect of the movable platform when performing operations using the working path can be guaranteed. Specifically, the working path can be generated on the reference surface, or there can be a gap between the working path and the reference surface. The specific gap can be set according to actual needs.

[0316] In some embodiments, the path planning method further includes: adjusting the reference surface based on the positional relationship between the reference surface and the target object in the image so that the adjusted reference surface coincides with the target surface or the deviation between the adjusted reference surface and the target surface is less than a preset value; displaying the adjusted reference surface in the image; and generating an operating path for the movable platform based on the adjusted reference surface. This embodiment automatically adjusts the reference surface based on the positional relationship between the reference surface and the target object in the image so that the adjusted reference surface coincides with the target surface or the deviation between the adjusted reference surface and the target surface is less than a preset value, thereby ensuring consistency between the reference surface and the target surface. In this way, the subsequent generation of the operating path for the movable platform based on the adjusted reference surface can ensure the matching degree between the operating path and the target surface, thereby ensuring the operating effect when the movable platform uses the operating path to perform operations.

[0317] In some embodiments, the path planning method further includes: in response to a user's adjustment operation on the reference surface, adjusting the reference surface so that the adjusted reference surface coincides with the target surface or the deviation between the adjusted reference surface and the target surface is less than a preset value; displaying the adjusted reference surface in the image; and generating an operating path of the movable platform based on the adjusted reference surface.

[0318] In this embodiment, the user manually adjusts the reference surface so that the adjusted reference surface coincides with the target surface or the deviation between the adjusted reference surface and the target surface is less than a preset value, thereby ensuring the consistency between the reference surface and the target surface. In this way, the operation path of the movable platform is subsequently generated based on the adjusted reference surface, which can ensure the matching degree between the operation path and the target surface, thereby ensuring the operation effect when the movable platform uses the operation path to perform operations.

[0319] In some embodiments, the deviation between the reference surface and the target surface is less than a preset value and may include at least one of the following: the distance between the reference surface and the target surface in the direction of the Z axis of the preset three-dimensional coordinate system is less than a first preset value, the plane formed by the X axis and the Y axis of the preset three-dimensional coordinate system is parallel to the reference surface, and the Z axis is perpendicular to the reference surface; the inclination angle between the reference surface and the target surface is less than a second preset value, and the inclination angle is the angle at which the reference surface is tilted relative to the target surface around the X axis or the Y axis; the distance between the contour of the reference surface and the contour of the target surface is less than a third preset value; the deviation angle between the reference surface and the target surface in the direction around the Z axis is less than a fourth preset value.

[0320] In some embodiments, adjusting the reference surface includes adjusting geometric information of the reference surface, wherein the geometric information of the reference surface includes at least one of the following: the area of ​​the reference surface, the shape of the reference surface, and the tilt angle of the reference surface.

[0321] Specifically, the user's adjustment operation on the reference surface includes at least one of a first operation and a second operation, the first operation is used to adjust the area size and / or shape of the reference surface, and the second operation is used to adjust the inclination angle of the reference surface and / or the distance between the reference surface and the target surface. For example, the reference surface includes at least one adjustment point located on the boundary of the reference surface, and the user's adjustment operation on the reference surface includes the user dragging or clicking the adjustment point. For another example, the reference surface includes multiple endpoint adding buttons and / or endpoint deleting buttons, and the user's adjustment operation on the reference surface includes the user dragging or clicking the endpoint adding buttons and / or endpoint deleting buttons. Among them, at least one endpoint adding button and / or endpoint deleting button is displayed on each boundary of the reference surface. For another example, the user can move the reference surface to adjust the distance between the reference surface and the target surface, or rotate the reference surface to adjust the angle of the reference surface.

[0322] In some embodiments, in response to a user's adjustment operation on the reference surface, adjusting the reference surface may include: adjusting the reference surface in the live view screen in response to the user's adjustment operation on the reference surface in the live view screen. Alternatively, in response to the user's adjustment operation on the reference surface in the overview view, adjusting the reference surface in the overview view, wherein the overview view displays a target surface and a reference surface of a target object viewed at a preset angle by a movable platform. When the user's adjustment operation on the reference surface is obtained in the live view screen, the reference surface in the overview view also changes in response to the user's adjustment operation on the reference surface; and / or when the user's adjustment operation on the reference surface is obtained in the overview view, the reference surface in the live view screen also changes in response to the user's adjustment operation on the reference surface.

[0323] In some embodiments, after generating the operation path, the path planning method provided in the embodiments of the present application further includes: adjusting the operation path based on the positional relationship between the operation path and each object in the image, and displaying the adjusted operation path in the image. The positional relationship between the operation path and each object in the image can be determined based on a pre-generated perception map, and the adjusted operation path has no intersection with each object in the image. This embodiment automatically adjusts the operation path based on the positional relationship between the operation path and each object in the image, so that the adjusted operation path has no intersection with each object in the image, thereby ensuring the safety of the movable platform during operation.

[0324] In some embodiments, after generating the operation path, the path planning method provided in embodiments of the present application further comprises: adjusting the operation path in response to a user's adjustment operation on the operation path, and displaying the adjusted operation path in the image. The adjusted operation path does not intersect with any object in the image. This embodiment ensures the safety of the movable platform during operation by adjusting the operation path so that the adjusted operation path does not intersect with any object in the image.

[0325] In some embodiments, after generating the work path, the path planning method provided in embodiments of the present application further includes: determining, based on the positional relationship between the work path and each object in the image, path segments in the work path that have a collision risk, and marking the path segments that have a collision risk. This embodiment automatically determines and marks the path segments in the work path that have a collision risk based on the positional relationship between the work path and each object in the image, so that the user is aware of the collision risk of the work path and can adjust the work path in a timely manner.

[0326] In some embodiments, the path planning method provided by the embodiments of the present application further includes: adaptively adjusting the operation path in response to the user's operation parameter adjustment operation. The user's operation parameter adjustment operation includes the user's operation parameter adjustment operation in the real-time view screen and / or the user's operation parameter adjustment operation in the overview map. When the operation parameter adjustment operation is obtained in the real-time view screen, the operation path in the overview map also changes in response to the operation parameter adjustment operation; and / or when the operation parameter adjustment operation is obtained in the overview map, the operation path in the real-time view screen also changes in response to the operation parameter adjustment operation. This embodiment can realize the adjustment of the operation parameters based on the real-time view screen and / or the overview map, so that the user can choose the appropriate adjustment method to adjust the operation parameters according to actual needs, thereby improving the user experience.

[0327] It should be noted that the path planning method described in this embodiment is the same or similar in structure and principle to the path planning methods of all the aforementioned embodiments. Those skilled in the art can clearly know that one or more implementation methods of the aforementioned embodiments can be applied to the path planning method of this embodiment, and this embodiment will not be described in detail here.

[0328] Please refer to Figure 23, which is a flowchart showing the steps of a control method provided by an embodiment of the present application. The control method can be applied to a control terminal.

[0329] As shown in FIG. 23 , the control method includes steps S501 to S506 .

[0330] Step S501: In response to a user operation, the movable platform is controlled to move to the vicinity of a target object, wherein the movable platform is equipped with an image sensor and a distance measuring sensor.

[0331] In this embodiment, the target object is any object in real three-dimensional space. The target object can be set by the user and is not specifically limited in this embodiment of the present application. The target object can be an object that needs to be surveyed, photographed, sprayed, spread, placed, or cleaned, such as a building, a mountain, a river, or a farmland. The user can manually control the movable platform to move to the vicinity of the target object, or set a motion path for the movable platform to the target object and control the movable platform to move to the vicinity of the target object based on the motion path.

[0332] Step S502: Display the image of the target surface including the target object acquired by the image sensor.

[0333] In some embodiments, the target surface and the target working area in the aforementioned embodiments may have the same meaning and may refer to the same object or area.

[0334] In this embodiment, after the movable platform moves to the vicinity of the target object, the image sensor onboard the movable platform can capture an image containing the target surface of the target object. The movable platform transmits the image containing the target surface captured by the image sensor to the control terminal, which then displays the image containing the target surface captured by the image sensor. The target surface and the target operation area in the aforementioned embodiment may refer to the same area.

[0335] Step S503: Determine a reference surface in the image based at least on the sensing information of the ranging sensor, and associate position information of the reference surface with position information of the target surface.

[0336] In this embodiment, the target object can be any object in the image, any object that meets the preset saliency conditions, or an object selected by a pre-selection box in the image, and this embodiment of the application does not specifically limit this. The target surface of the target object can be one of the surfaces of the target object.

[0337] In some embodiments, the reference surface and the virtual working area in the aforementioned embodiments may have the same meaning and may refer to the same object or area.

[0338] In some embodiments, the sensing information of the ranging sensor includes the distance and / or ranging direction of the target surface of the target object relative to the movable platform. The position information of the reference surface can be obtained through the sensing information of the ranging sensor and the geographical location information of the movable platform itself.

[0339] In some embodiments, determining a reference surface in an image based at least on sensing information from a ranging sensor includes:

[0340] determining the distance between the target surface and the movable platform based on information acquired by the ranging sensor; and

[0341] The geographic location information of the reference surface is determined based on the distance between the target surface and the movable platform, so that the reference surface is aligned with the target surface as closely as possible. It should be noted that the distance between the target surface and the movable platform can refer to the distance between the target surface and the movable platform itself, or it can also refer to the distance between the target surface and a payload on the movable platform itself (such as an image sensor or a ranging sensor, etc.).

[0342] In some embodiments, determining a reference surface in an image based at least on sensing information from a ranging sensor further includes:

[0343] The geographical location information of the reference surface is determined based on the distance direction between the target surface and the movable platform, so that the reference surface fits the target surface as closely as possible.

[0344] Specifically, when the sensing information of the ranging sensor is the ranging information of a representative point on the target surface, the position information of the reference point corresponding to the representative point on the reference surface can be obtained through the sensing information of the ranging sensor and the geographic location information of the movable platform itself. Based on the geographic location information of the reference point corresponding to the representative point and the preset geometric dimensions of the reference surface, the geographic location information of the reference surface can be determined. When the sensing information of the ranging sensor is the ranging information of multiple feature points on the target surface, the position information of the reference points corresponding to the multiple feature points on the reference surface can be obtained through the sensing information of the ranging sensor and the geographic location information of the movable platform itself, and the reference surface can be automatically generated based on the position information of the multiple reference points. It should be noted that the distance between the target surface and the movable platform can refer to the distance between the target surface and the movable platform body, or it can refer to the distance between the target surface and the load on the movable platform body (such as an image sensor or a ranging sensor, etc.).

[0345] In some embodiments, when the sensing information of the ranging sensor is the ranging information of a representative feature point on the target surface, a virtual working area may be generated in the live view screen based on the pre-selected box and the sensing information of the ranging sensor.

[0346] The size of the virtual surface is determined based on the pre-selected frame, and the position of the virtual working area is determined based on the sensing information of the ranging sensor.

[0347] In some embodiments, when the sensing information of the ranging sensor is the ranging information of a representative feature point on the target surface, the reference surface can be determined in the image based on the geometric information of the reference surface and the sensing information of the ranging sensor. The geometric information of the target surface is substantially the same as the geometric information of the reference surface. The geometric information of the target surface includes the area, shape, and / or tilt angle of the target surface.

[0348] In some embodiments, the path planning method provided by the embodiments of the present application further includes: selecting one of the surfaces of the target object in the image as the target surface. For example, in response to a user selecting a surface of the target object, selecting one of the surfaces of the target object in the image as the target surface.

[0349] In some embodiments, the image displayed by the control terminal is a real image, and the reference surface is a virtual image, or the image displayed by the control terminal and the reference surface are virtual images.

[0350] Step S504 : In response to the user's adjustment operation on the reference surface, the reference surface is adjusted in the image to obtain an adjusted reference surface.

[0351] In some embodiments, in response to a user adjustment operation on the reference surface, adjusting the reference surface in the image to obtain an adjusted reference surface may include: adjusting geometric information of the reference surface in the image in response to the user adjustment operation on the reference surface. The geometric information of the reference surface includes at least one of the following: an area of ​​the reference surface, a shape of the reference surface, and an inclination angle of the reference surface.

[0352] In some embodiments, adjusting the reference surface in the image to obtain the adjusted reference surface includes: adjusting the reference surface based on the positional relationship between the reference surface and the target object in the image so that the adjusted reference surface coincides with the target surface or the deviation between the adjusted reference surface and the target surface is less than a preset value. This embodiment automatically adjusts the reference surface based on the positional relationship between the reference surface and the target object in the image so that the adjusted reference surface coincides with the target surface or the deviation between the adjusted reference surface and the target surface is less than a preset value, thereby ensuring the consistency between the reference surface and the target surface. In this way, the subsequent generation of the operating path of the movable platform based on the adjusted reference surface can ensure the matching degree between the operating path and the target surface, thereby ensuring the operating effect when the movable platform uses the operating path to perform operations.

[0353] In some scenarios, it is impossible to accurately know the relative position relationship between the target surface and the reference surface, such as errors in the measured distance, which makes it impossible to directly obtain the geographic location information of the target surface. Therefore, through the pre-generated reference surface, the geographic location information of the reference surface can be accurately known, and through adjustment operations, the change in geographic location information corresponding to the adjustment operation can also be determined according to preset rules. Therefore, through the virtual operation area and adjustment operations, when the user visually determines in the real-time view screen that the virtual operation area coincides with the target operation area or the deviation between the virtual operation area and the target operation area is less than the preset value, the geographic location information of the target operation area can be inferred.

[0354] In some embodiments, adjusting the reference surface in the image to obtain an adjusted reference surface includes: adjusting the reference surface in response to a user's adjustment operation on the reference surface so that the adjusted reference surface coincides with the target surface or the deviation between the adjusted reference surface and the target surface is less than a preset value; displaying the adjusted reference surface in the image; and generating a working path for the movable platform based on the adjusted reference surface. In this embodiment, the user manually adjusts the reference surface so that the adjusted reference surface coincides with the target surface or the deviation between the adjusted reference surface and the target surface is less than a preset value, thereby ensuring consistency between the reference surface and the target surface. In this way, the subsequent generation of the working path for the movable platform based on the adjusted reference surface can ensure a match between the working path and the target surface, thereby ensuring the working effect when the movable platform uses the working path.

[0355] In some embodiments, the deviation between the reference surface and the target surface is less than a preset value and may include at least one of the following: the distance between the reference surface and the target surface in the direction of the Z axis of the preset three-dimensional coordinate system is less than a first preset value, the plane formed by the X axis and the Y axis of the preset three-dimensional coordinate system is parallel to the reference surface, and the Z axis is perpendicular to the reference surface; the inclination angle between the reference surface and the target surface is less than a second preset value, and the inclination angle is the angle at which the reference surface is tilted relative to the target surface around the X axis or the Y axis; the distance between the contour of the reference surface and the contour of the target surface is less than a third preset value; the deviation angle between the reference surface and the target surface in the direction around the Z axis is less than a fourth preset value.

[0356] In this embodiment, the user's adjustment operation on the reference surface includes at least one of a first operation and a second operation, the first operation is used to adjust the area size and / or shape of the reference surface, and the second operation is used to adjust the inclination angle of the reference surface and / or the distance between the reference surface and the target surface.

[0357] In some embodiments, the reference surface includes at least one adjustment point located on the boundary of the reference surface, and the user's adjustment operation on the reference surface includes the user dragging or clicking the adjustment point. In other embodiments, the reference surface includes multiple endpoint addition buttons and / or endpoint deletion buttons, and the user's adjustment operation on the reference surface includes the user dragging or clicking the endpoint addition buttons and / or endpoint deletion buttons. At least one endpoint addition button and / or endpoint deletion button is displayed on each boundary of the reference surface. For another example, the user can move the reference surface to adjust the distance between the reference surface and the target surface, or rotate the reference surface to adjust the angle of the reference surface.

[0358] In some embodiments, in response to a user's adjustment operation on the reference surface, adjusting the reference surface in the image to obtain an adjusted reference surface may include: in response to the user's adjustment operation on the reference surface in the live view screen, adjusting the reference surface in the live view screen to obtain the adjusted reference surface. Alternatively, in response to the user's adjustment operation on the reference surface in the overview image, adjusting the reference surface in the overview image to obtain the adjusted reference surface, the overview image displaying a target surface and a reference surface of a target object with a movable platform at a preset viewing angle. When the user's adjustment operation on the reference surface is obtained in the live view screen, the reference surface in the overview image also changes in response to the user's adjustment operation on the reference surface; and / or when the user's adjustment operation on the reference surface is obtained in the overview image, the reference surface in the live view screen also changes in response to the user's adjustment operation on the reference surface.

[0359] In some embodiments, before step S504, the control method provided in embodiments of the present application further includes: changing the displayed image, where the changed image has a different viewing angle than the image before the change; and continuing to display the reference surface in the changed image, where the changed image includes the target surface and the reference surface at the other viewing angle. This embodiment changes the image so that the changed image includes the target surface and the reference surface of the target object at the other viewing angle, thereby facilitating the user's visual inspection of the deviation between the reference surface and the target surface at the other viewing angle, thereby improving the ease and accuracy of adjusting the reference surface.

[0360] Step S505: Generate an operation path of the movable platform based on the adjusted reference surface.

[0361] In some embodiments, at least a portion of the working path is generated on the adjusted reference surface, or at least a portion of the working path is substantially parallel or parallel to the adjusted reference surface.

[0362] In some embodiments, after generating the work path, the control method provided in the embodiments of the present application further includes: adjusting the work path based on the positional relationship between the work path and each object in the image, and displaying the adjusted work path in the image. The positional relationship between the work path and each object in the image can be determined based on a pre-generated perception map, and the adjusted work path has no intersection with each object in the image. This embodiment automatically adjusts the work path based on the positional relationship between the work path and each object in the image, so that the adjusted work path has no intersection with each object in the image, thereby ensuring the safety of the movable platform during operation.

[0363] After generating the work path, the control method provided in an embodiment of the present application further includes: adjusting the work path in response to a user's adjustment operation on the work path, and displaying the adjusted work path in the image. The adjusted work path does not intersect with any object in the image. This embodiment ensures the safety of the movable platform during operation by adjusting the work path so that the adjusted work path does not intersect with any object in the image.

[0364] In some embodiments, after generating the work path, the control method provided by the embodiments of the present application further includes: determining, based on the positional relationship between the work path and each object in the image, path segments in the work path that have a collision risk, and marking the path segments that have a collision risk. This embodiment automatically determines and marks the path segments in the work path that have a collision risk based on the positional relationship between the work path and each object in the image, so that the user is aware of the collision risk of the work path and can adjust the work path in a timely manner.

[0365] Step S506: Control the movable platform to move along the operation path.

[0366] The entire implementation process of the control method provided in this embodiment does not require determining the measurement area based on the marking of points on the movable platform, thereby reducing the complexity of path planning and improving the efficiency of path planning for the movable platform. Furthermore, after the path is planned, the movable platform can be immediately controlled to perform operations near the target object without waiting, thereby improving the real-time performance of the operation. Simultaneously, based on the sensing information of at least the ranging sensor, a reference surface is determined in the image, and the reference surface is adjusted so that the adjusted reference surface can be set according to user needs, such as being adjusted to align with the target surface of the target object in the image. Therefore, based on the adjusted reference surface, an operating path for the movable platform is generated, ensuring the matching degree between the operating path and the target surface of the target object, thereby ensuring the operating effect when the movable platform uses the operating path.

[0367] In some embodiments, the control method provided by the embodiment of the present application further includes: adaptively adjusting the operation path in response to the user's operation parameter adjustment operation. The user's operation parameter adjustment operation includes the user's operation parameter adjustment operation in the real-time view screen and / or the user's operation parameter adjustment operation in the overview diagram. When the operation parameter adjustment operation is obtained in the real-time view screen, the operation path in the overview diagram also changes in response to the operation parameter adjustment operation; and / or when the operation parameter adjustment operation is obtained in the overview diagram, the operation path in the real-time view screen also changes in response to the operation parameter adjustment operation. This embodiment can realize the adjustment of the operation parameters based on the real-time view screen and / or the overview diagram, so that the user can choose the appropriate adjustment method to adjust the operation parameters according to actual needs, thereby improving the user experience.

[0368] It should be noted that the control method described in this embodiment is the same or similar in structure and principle to the path planning methods of all the aforementioned embodiments. Those skilled in the art can clearly know that one or more implementation methods of the aforementioned embodiments can be applied to the control method of this embodiment, and this embodiment will not be described in detail here.

[0369] Please refer to Figure 24, which is a schematic block diagram of the structure of a path planning device provided in an embodiment of the present application. The path planning device can be applied to a control terminal or a mobile platform.

[0370] As shown in FIG24 , the path planning device 300 includes a processor 310 and a memory 320 . The processor 310 and the memory 320 are connected via a bus 330 , which is, for example, an I2C (Inter-Integrated Circuit) bus.

[0371] Specifically, the processor 310 may be a microcontroller unit (MCU), a central processing unit (CPU), or a digital signal processor (DSP).

[0372] Specifically, the memory 320 may be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a mobile hard disk.

[0373] The processor 310 is configured to run a computer program stored in the memory 320 and implement the following steps when executing the computer program:

[0374] Determine the target operating area;

[0375] Automatically generating a virtual work area on a real-time view screen of a mobile platform according to preset conditions, wherein geographical location information corresponding to the virtual work area in real three-dimensional space is determined based on the preset conditions;

[0376] Adjusting the virtual operating area according to the target operating area; and

[0377] Plan the operation path based on the adjusted virtual operation area.

[0378] In some embodiments, the preset condition includes: determining the position of the virtual working area based on the relative position relationship between the target working area and the movable platform.

[0379] In some embodiments, determining the position of the virtual work area based on the relative positional relationship between the target work area and the movable platform includes:

[0380] The position of the virtual work area is determined based on the distance between the target work area and the movable platform.

[0381] In some embodiments, the distance between the position corresponding to the virtual work area in the real three-dimensional space and the movable platform is equal to or approximately equal to the distance between the target work area and the movable platform; or, there is a preset difference between the distance between the position corresponding to the virtual work area in the real three-dimensional space and the movable platform and the distance between the target work area and the movable platform.

[0382] In some embodiments, the distance between the movable platform and the target operating area is measured by a ranging sensor; or the distance between the target operating area and the movable platform is determined based on a pre-generated perception map and the geographic location information of the movable platform itself.

[0383] In some embodiments, determining the position of the virtual work area based on the relative positional relationship between the target work area and the movable platform includes:

[0384] The orientation of the virtual work area is determined based on a distance direction, where the distance direction is the direction along which the distance between the movable platform and the target work area is determined; or, the orientation of the virtual work area is determined based on a viewing direction of the movable platform.

[0385] In some embodiments, when the orientation of the virtual work area is determined based on the fixed distance direction, the projection direction of the virtual work area is consistent with the fixed distance direction, or an angle is set between the projection direction of the virtual work area and the fixed distance direction; when the orientation of the virtual work area is determined based on the viewing direction of the movable platform, the projection direction of the virtual work area is consistent with the viewing direction, or an angle is set between the projection direction of the virtual work area and the viewing direction.

[0386] In some embodiments, the viewing direction of the movable platform is consistent with the distance direction.

[0387] In some embodiments, the viewing direction of the movable platform is perpendicular or approximately perpendicular to the target working area, so that the generated virtual working area is parallel or approximately parallel to the target working area.

[0388] In some embodiments, before the step of determining the target operation area, the method further includes:

[0389] Obtaining an adjustment instruction from a user, where the adjustment instruction is used to adjust the viewing angle of the movable platform; and

[0390] According to the adjustment instruction, the viewing angle of the movable platform is adjusted so that the viewing direction of the movable platform is perpendicular or approximately perpendicular to the target operation area; or

[0391] determining the angle of the target operating area based on a pre-generated perception map; and

[0392] Based on the angle of the target working area, the viewing angle of the movable platform is automatically determined so that the viewing direction of the movable platform is perpendicular or approximately perpendicular to the target working area.

[0393] In some embodiments, determining the position of the virtual work area based on the relative positional relationship between the target work area and the movable platform includes:

[0394] The position of the virtual work area is determined based on the orientation of the target work area relative to the movable platform.

[0395] In some embodiments, the processor is further configured to implement:

[0396] In response to the viewing direction of the movable platform facing the target working area and in response to the target working area being determined, a virtual working area is automatically generated in the real-time viewing picture of the movable platform according to preset conditions.

[0397] In some embodiments, before the step of determining the target operation area, a pre-selection box is displayed on the real-time view screen, and the pre-selection box is used to indicate that the area selected on the real-time view screen is the target operation area;

[0398] The step of automatically generating a virtual operating area on the real-time viewing screen of the movable platform according to preset conditions includes:

[0399] The virtual operation area is automatically generated in the live view image based on the pre-selected box and the preset conditions.

[0400] In some embodiments, the step of generating the virtual operating area in the live view image based on the pre-selected box and the preset conditions includes:

[0401] The size of the virtual operation area in the live view image is determined based on the pre-selected frame, and the position of the virtual operation area is determined based on the preset condition.

[0402] In some embodiments, the size of the pre-selection box in the live view image is a preset size.

[0403] In some embodiments, before the step of determining the target operating area, the method further includes:

[0404] Displaying the pre-selected frame on the live view screen; and

[0405] A framing adjustment operation is acquired, and according to the framing adjustment operation, a framing direction or size is adjusted so that the pre-selected frame at least partially covers the target operation area.

[0406] In some embodiments, the step of automatically generating a virtual work area in the real-time view image of the movable platform according to preset conditions includes:

[0407] Extracting a contour line of the target operating area in the real-time view image according to the determined target operating area; and

[0408] The virtual operation area is automatically generated in the live view image based on the area defined by the contour line and the preset conditions.

[0409] In some embodiments, the step of generating the virtual operating area in the live view image based on the area defined by the contour line and the preset conditions includes:

[0410] The size of the virtual work area in the live view image is determined based on the area delimited by the contour line, and the position of the virtual work area is determined based on the preset condition.

[0411] In some embodiments, the step of automatically generating a virtual work area in the real-time view image of the movable platform according to preset conditions includes:

[0412] Determining a plurality of feature points according to the determined target operation area;

[0413] Obtaining a relative positional relationship between each of the feature points and the movable platform;

[0414] Determining the geographical location information of each of the feature points based on the geographical location information of the movable platform and the relative position relationship; and

[0415] The virtual operation area is automatically generated based on the geographical location information of each feature point.

[0416] In some embodiments, the step of generating the virtual operating area based on the geographic location information of each feature point includes:

[0417] According to the geographical location information of the feature point, a point mark is superimposed on the corresponding feature point;

[0418] The plurality of point marks are connected according to the actual relative positional relationship between the plurality of feature points, thereby generating and displaying the virtual operation area covering the target operation area.

[0419] In some embodiments, the feature points include at least endpoints of a boundary line of the target operation area.

[0420] In some embodiments, the relative position relationship includes a distance between the feature point and the movable platform, and a measurement direction when measuring the distance between the feature point and the movable platform.

[0421] In some embodiments, the step of determining a plurality of feature points includes:

[0422] Obtain the user's operation of determining multiple feature points; or,

[0423] Automatically extract multiple feature points.

[0424] In some embodiments, the step of adjusting the virtual operation area according to the target operation area includes: adjusting the virtual operation area according to the target operation area so that the adjusted virtual operation area coincides with the target operation area or the deviation between the adjusted virtual operation area and the target operation area is less than a preset value.

[0425] In some embodiments, in the step of adjusting the virtual operation area according to the target operation area, the user visually determines whether the virtual operation area and the target operation area overlap or whether the deviation between the virtual operation area and the target operation area is less than the preset value.

[0426] In some embodiments, the deviation between the virtual operating area and the target operating area is less than a preset value, including at least one of the following:

[0427] The distance between the virtual work area and the target work area in the Z-axis direction of the preset three-dimensional coordinate system is less than a first preset value, the plane formed by the X-axis and the Y-axis of the preset three-dimensional coordinate system is parallel to the virtual work area, and the Z-axis is perpendicular to the virtual work area;

[0428] The tilt angle between the virtual operating area and the target operating area is less than a second preset value, the tilt angle being the angle at which the virtual operating area is tilted relative to the target operating area around the X-axis or the Y-axis;

[0429] The distance between the outline of the virtual operating area and the outline of the target operating area is less than a third preset value;

[0430] A deviation angle between the virtual operating area and the target operating area in the Z-axis direction is smaller than a fourth preset value.

[0431] In some embodiments, the step of adjusting the virtual operating area according to the target operating area includes at least a first operation and / or a second operation; wherein,

[0432] The first operation is used to adjust at least one of the size, shape, position and orientation of the virtual work area;

[0433] The second operation is used to adjust the distance between the virtual working area and the target working area and / or the angle of the virtual working area relative to the target working area.

[0434] In some embodiments, the step of adjusting the virtual operating area according to the target operating area includes:

[0435] Adjusting the movable platform so that the viewing direction of the movable platform is switched to another angle of the target work area, and the real-time viewing screen presents the target work area and the virtual work area at the other angle, wherein the other angle is different from the viewing angle of the movable platform when generating the virtual work area;

[0436] The virtual operating area is adjusted according to the target operating area at the other angle and the virtual operating area.

[0437] In some embodiments, the target operation area is a planar area, and adjusting the movable platform so that the viewing direction of the movable platform is switched to another angle of the target operation area includes:

[0438] Adjusting the movable platform so that the viewing direction of the movable platform is switched to the side of the target working area, and the real-time viewing image includes the side of the target working area and the side of the virtual working area;

[0439] The adjusting the virtual operating area according to the target operating area at the other angle and the virtual operating area to obtain an adjusted virtual operating area includes:

[0440] When the viewing direction of the movable platform is switched to the side of the target working area, the distance between the virtual working area and the target working area is adjusted and / or the angle of the virtual working area relative to the target working area is adjusted.

[0441] In some embodiments, before determining the target operating area, the method further includes:

[0442] The movable platform is adjusted so that the viewing direction of the movable platform is the front of the target operation area, and the real-time viewing picture at least includes the front part of the target operation area.

[0443] In some embodiments, the step of adjusting the virtual work area according to the target work area includes: adjusting at least one of the size, shape, position and orientation of the virtual work area according to the target work area in front.

[0444] In some embodiments, the step of adjusting the virtual operating area according to the target operating area includes:

[0445] adjusting the virtual operation area based on an acquired area adjustment operation of the user, wherein the user adjusts the virtual operation area according to the target operation area; or

[0446] The virtual operating area is automatically adjusted according to the target operating area and a preset area adjustment algorithm.

[0447] In some embodiments, the step of obtaining the user's region adjustment operation includes at least one of the following:

[0448] Acquiring the area adjustment operation in the live view picture; or

[0449] The area adjustment operation is obtained in an overview image, wherein the overview image displays the target operation area and the virtual operation area viewed by the movable platform at a preset angle.

[0450] In some embodiments, when the area adjustment operation is acquired in the live view image, the virtual operation area in the overview image also changes in response to the area adjustment operation; and / or,

[0451] When the area adjustment operation is acquired in the overview image, the virtual work area in the live view screen also changes in response to the area adjustment operation.

[0452] In some embodiments, the processor is further configured to implement:

[0453] The overview map is automatically generated in response to the generation of the virtual work area; or, the overview map is obtained in response to a user's photographing operation.

[0454] In some embodiments, the virtual operation area includes at least one adjustment point located on the boundary of the virtual operation area, and the area adjustment operation includes a user dragging or clicking operation on the adjustment point; and / or, the virtual operation area includes multiple endpoint addition buttons and / or endpoint deletion buttons, and the area adjustment operation includes a user dragging or clicking operation on the endpoint addition button and / or the endpoint deletion button.

[0455] In some embodiments, the path planning method is applied to a control terminal, the control terminal is configured to be communicatively connected with the movable platform, and the control terminal is configured to be able to display the real-time view picture.

[0456] In some embodiments, the processor is further configured to implement:

[0457] The movable platform is controlled to operate according to the operation path.

[0458] In some embodiments, the step of planning the operation path based on the adjusted virtual operation area includes:

[0459] generating the operation path based on the adjusted virtual operation area; or

[0460] An initial operation path is generated based on the virtual operation area, and based on the adjustment of the virtual operation area, the initial operation path is adaptively adjusted, thereby obtaining the operation path planned based on the adjusted virtual operation area.

[0461] In some embodiments, the step of planning the operation path includes:

[0462] The "bow"-shaped or "Z"-shaped operation path is automatically generated based on the virtual operation area, the operation parameters set by the user and the preset planning rules.

[0463] In some embodiments, the step of adjusting the virtual operating area according to the target operating area further includes:

[0464] Based on the obtained geographical location information of the virtual operating area and the amount of change in the geographical location information corresponding to the adjustment of the virtual operating area, the adjusted geographical location information of the virtual operating area is determined.

[0465] In some embodiments, the processor is further configured to implement:

[0466] The geographical location information of the virtual work area is determined based on the geographical location information of the movable platform and the preset condition.

[0467] In some embodiments, the amount of change in the geographical location information corresponding to the virtual operating area is adjusted according to a preset rule.

[0468] In some embodiments, the processor is further configured to: determine the target work area in the displayed image.

[0469] In some embodiments, the step of determining the target operating area includes at least one of the following:

[0470] Get the user's selection operation on the target operation area; or

[0471] The target operating area is automatically determined according to preset target operating area determination conditions.

[0472] In some embodiments, the movable platform includes an aerial photography aircraft, a ground viewing device, a mobile scanning device, a mobile measuring device, a mobile mapping device, a camera, or a mobile phone.

[0473] In some embodiments, the target operation area is a planar operation area.

[0474] In some embodiments, the target operation area includes at least one of the following:

[0475] Vertical working area, inclined working area or flat working area.

[0476] In some embodiments, the real-time view image displays the distance between the target working area and the movable platform.

[0477] In some embodiments, the processor is further configured to implement:

[0478] When the distance between the target operation area and the movable platform exceeds a preset distance range, a prompt message is issued.

[0479] In some embodiments, the processor is further configured to implement:

[0480] Determining whether there is a path segment in the operation path that collides with an obstacle;

[0481] If there is a path segment that collides with an obstacle in the operation path, the path segment that collides with the obstacle is marked in the operation path.

[0482] In some other embodiments, the processor is configured to run a computer program stored in the memory, and implement the following steps when executing the computer program:

[0483] Determine the target operating area;

[0484] determining a distance between the target operating area and the movable platform;

[0485] generating a virtual work area on a real-time viewing screen of the movable platform based on parameters related to the distance; and

[0486] Based on the virtual operation area, an operation path is planned.

[0487] In some embodiments, the virtual operating area is superimposed on the target operating area.

[0488] In some embodiments, the parameters related to the distance include: the distance and / or the distance direction between the target working area and the movable platform, wherein the distance between the target working area and the movable platform is determined along the distance direction.

[0489] In some embodiments, the step of generating a virtual work area on the real-time view screen of the movable platform based on the parameters related to the distance includes:

[0490] Based on the distance between the target work area and the movable platform, the distance between the virtual work area and the movable platform is determined.

[0491] In some embodiments, the distance between the position corresponding to the virtual work area in the real three-dimensional space and the movable platform is equal to or approximately equal to the distance between the target work area and the movable platform; or, there is a preset difference between the distance between the position corresponding to the virtual work area in the real three-dimensional space and the movable platform and the distance between the target work area and the movable platform.

[0492] In some embodiments, the distance between the movable platform and the target operating area is measured by a ranging sensor; or the distance between the target operating area and the movable platform is determined based on a pre-generated perception map and the geographic location information of the movable platform itself.

[0493] In some embodiments, the step of generating a virtual work area on the real-time view screen of the movable platform based on the parameters related to the distance further includes:

[0494] The orientation of the virtual work area is determined based on the distance direction, or the orientation of the virtual work area is determined based on the viewing direction of the movable platform.

[0495] In some embodiments, when the orientation of the virtual work area is determined based on the fixed distance direction, the projection direction of the virtual work area is consistent with the fixed distance direction, or an angle is set between the projection direction of the virtual work area and the fixed distance direction; when the orientation of the virtual work area is determined based on the viewing direction of the movable platform, the projection direction of the virtual work area is consistent with the viewing direction, or an angle is set between the projection direction of the virtual work area and the viewing direction.

[0496] In some embodiments, the viewing direction of the movable platform is consistent with the distance direction.

[0497] In some embodiments, the viewing direction of the movable platform is perpendicular or approximately perpendicular to the target working area, so that the generated virtual working area is parallel or approximately parallel to the target working area.

[0498] In some embodiments, before the step of determining the target operating area, the method further includes:

[0499] Obtaining an adjustment instruction from a user, where the adjustment instruction is used to adjust the viewing angle of the movable platform; and

[0500] According to the adjustment instruction, the viewing angle of the movable platform is adjusted so that the viewing direction of the movable platform is perpendicular or approximately perpendicular to the target operation area; or

[0501] determining the angle of the target operating area based on a pre-generated perception map; and

[0502] Based on the angle of the target working area, the viewing angle of the movable platform is automatically determined so that the viewing direction of the movable platform is perpendicular or approximately perpendicular to the target working area.

[0503] In some embodiments, before the step of determining the target operation area, a pre-selection box is displayed on the real-time view screen, and the pre-selection box is used to indicate that the area selected on the real-time view screen is the target operation area;

[0504] The step of generating a virtual operating area on the real-time view screen of the movable platform based on the parameters related to the distance includes:

[0505] The virtual work area is generated in the live view screen based on the parameters related to the distance and the pre-selected box.

[0506] In some embodiments, the step of generating the virtual operating area in the live view image based on the parameters related to the distance and the pre-selected box includes:

[0507] The size of the virtual work area in the live view image is determined based on the pre-selected frame, and the position corresponding to the virtual work area in the real three-dimensional space is determined based on parameters related to the distance.

[0508] In some embodiments, before the step of determining the target operating area, the processor is further configured to:

[0509] Displaying the pre-selected frame on the live view screen; and

[0510] A framing adjustment operation is acquired, and according to the framing adjustment operation, a framing direction or size is adjusted so that the pre-selected frame at least partially covers the target operation area.

[0511] In some embodiments, the step of generating a virtual work area on the real-time view screen of the movable platform based on the parameters related to the distance includes:

[0512] Extracting a contour line of the target operating area in the real-time view image according to the determined target operating area; and

[0513] The virtual work area is generated in the live view screen based on the area demarcated by the contour line and the determined parameters related to the distance.

[0514] In some embodiments, the step of generating the virtual working area in the live view image based on the area defined by the contour line and the determined parameters related to the distance includes:

[0515] The size of the virtual work area in the live view image is determined based on the area demarcated by the contour line, and the position corresponding to the virtual work area in the real three-dimensional space is determined based on parameters related to the distance.

[0516] In some embodiments, the steps of determining the distance between the target work area and the movable platform, and generating a virtual work area in a real-time view image of the movable platform based on parameters related to the distance, include:

[0517] Determining a plurality of feature points according to the determined target operation area;

[0518] Obtaining the distance between each of the feature points and the movable platform;

[0519] Determining the geographical location information of each of the feature points based on the geographical location information of the movable platform and the parameter related to the distance; and

[0520] The virtual operation area is generated based on the geographical location information of each feature point.

[0521] In some embodiments, the step of generating the virtual operating area based on the geographic location information of each feature point includes:

[0522] According to the geographical location information of the feature point, a point mark is superimposed on the corresponding feature point;

[0523] The plurality of point marks are connected according to the actual relative positional relationship between the plurality of feature points, thereby generating and displaying the virtual operation area covering the target operation area.

[0524] In some embodiments, the feature points include at least endpoints of a boundary line of the target operation area.

[0525] In some embodiments, the parameters related to the distance include the distance between the feature point and the movable platform, and a measurement direction when measuring the distance between the feature point and the movable platform.

[0526] In some embodiments, the step of determining a plurality of feature points includes:

[0527] Obtain the user's operation of determining multiple feature points; or,

[0528] Automatically extract multiple feature points.

[0529] In some embodiments, the processor is further configured to implement:

[0530] Adjusting the virtual operating area according to the target operating area; and

[0531] The operation path is planned based on the adjusted virtual operation area.

[0532] In some embodiments, the step of adjusting the virtual operation area according to the target operation area includes: adjusting the virtual operation area according to the target operation area so that the adjusted virtual operation area coincides with the target operation area or the deviation between the adjusted virtual operation area and the target operation area is less than a preset value.

[0533] In some embodiments, in the step of adjusting the virtual operation area according to the target operation area, the user visually determines whether the virtual operation area and the target operation area overlap or whether the deviation between the virtual operation area and the target operation area is less than the preset value.

[0534] In some embodiments, the deviation between the virtual operating area and the target operating area is less than a preset value, including at least one of the following:

[0535] The distance between the virtual work area and the target work area in the Z-axis direction of the preset three-dimensional coordinate system is less than a first preset value, the plane formed by the X-axis and the Y-axis of the preset three-dimensional coordinate system is parallel to the virtual work area, and the Z-axis is perpendicular to the virtual work area;

[0536] The tilt angle between the virtual operating area and the target operating area is less than a second preset value, the tilt angle being the angle at which the virtual operating area is tilted relative to the target operating area around the X-axis or the Y-axis;

[0537] The distance between the outline of the virtual operating area and the outline of the target operating area is less than a third preset value;

[0538] A deviation angle between the virtual operating area and the target operating area in the Z-axis direction is smaller than a fourth preset value.

[0539] In some embodiments, the step of adjusting the virtual operating area according to the target operating area includes at least a first operation and / or a second operation; wherein,

[0540] The first operation is used to adjust at least one of the size, shape, position and orientation of the virtual work area;

[0541] The second operation is used to adjust the distance between the virtual working area and the target working area and / or the angle of the virtual working area relative to the target working area.

[0542] In some embodiments, the step of adjusting the virtual operating area according to the target operating area includes:

[0543] Adjusting the movable platform so that the viewing direction of the movable platform is switched to another angle of the target work area, and the real-time viewing screen presents the target work area and the virtual work area at the other angle, wherein the other angle is different from the viewing angle of the movable platform when generating the virtual work area;

[0544] The virtual operating area is adjusted according to the target operating area at the other angle and the virtual operating area.

[0545] In some embodiments, the target operation area is a planar area, and adjusting the movable platform so that the viewing direction of the movable platform is switched to another angle of the target operation area includes:

[0546] Adjusting the movable platform so that the viewing direction of the movable platform is switched to the side of the target working area, and the real-time viewing image includes the side of the target working area and the side of the virtual working area;

[0547] The adjusting the virtual operating area according to the target operating area at the other angle and the virtual operating area to obtain an adjusted virtual operating area includes:

[0548] When the viewing direction of the movable platform is switched to the side of the target working area, the distance between the virtual working area and the target working area is adjusted and / or the angle of the virtual working area relative to the target working area is adjusted.

[0549] In some embodiments, before determining the target operating area, the processor is further configured to implement:

[0550] The movable platform is adjusted so that the viewing direction of the movable platform is the front of the target operation area, and the real-time viewing picture at least includes the front part of the target operation area.

[0551] In some embodiments, the step of adjusting the virtual work area according to the target work area includes: adjusting at least one of the size, shape, position and orientation of the virtual work area according to the target work area in front.

[0552] In some embodiments, the step of adjusting the virtual operating area according to the target operating area includes:

[0553] adjusting the virtual operation area based on an acquired area adjustment operation of the user, wherein the user adjusts the virtual operation area according to the target operation area; or

[0554] The virtual operating area is automatically adjusted according to the target operating area and a preset area adjustment algorithm.

[0555] In some embodiments, the step of obtaining the user's region adjustment operation includes at least one of the following:

[0556] Acquiring the area adjustment operation in the live view picture; or

[0557] The area adjustment operation is obtained in an overview image, wherein the overview image displays the target operation area and the virtual operation area viewed by the movable platform at a preset angle.

[0558] In some embodiments, when the area adjustment operation is acquired in the live view image, the virtual operation area in the overview image also changes in response to the area adjustment operation; and / or,

[0559] When the area adjustment operation is acquired in the overview image, the virtual work area in the live view screen also changes in response to the area adjustment operation.

[0560] In some embodiments, the method further comprises:

[0561] The overview map is automatically generated in response to the generation of the virtual work area; or, the overview map is obtained in response to a user's photographing operation.

[0562] In some embodiments, the virtual operation area includes at least one adjustment point located on the boundary of the virtual operation area, and the area adjustment operation includes a user dragging or clicking operation on the adjustment point; and / or, the virtual operation area includes multiple endpoint addition buttons and / or endpoint deletion buttons, and the area adjustment operation includes a user dragging or clicking operation on the endpoint addition button and / or the endpoint deletion button.

[0563] In some embodiments, a perception map is superimposed and displayed on the real-time view screen, and the perception map includes a perception model of the target operation area. The perception model is used to provide a reference basis for the area adjustment operation.

[0564] In some embodiments, the step of adjusting the virtual operating area according to the target operating area further includes:

[0565] Based on the obtained geographical location information of the virtual operating area and the amount of change in the geographical location information corresponding to the adjustment of the virtual operating area, the adjusted geographical location information of the virtual operating area is determined.

[0566] In some embodiments, the amount of change in the geographical location information corresponding to the virtual operating area is adjusted according to a preset rule.

[0567] In some embodiments, the processor is further configured to implement:

[0568] The geographical location information of the virtual work area is determined based on the geographical location information of the movable platform and a parameter related to the distance.

[0569] In some embodiments, the step of planning an operation path based on the virtual operation area includes:

[0570] generating the operation path based on the adjusted virtual operation area; or

[0571] An initial operation path is generated based on the virtual operation area, and based on the adjustment of the virtual operation area, the initial operation path is adaptively adjusted, thereby obtaining the operation path planned based on the adjusted virtual operation area.

[0572] In some embodiments, the step of planning the operation path includes:

[0573] The "bow"-shaped or "Z"-shaped operation path is automatically generated based on the virtual operation area, the operation parameters set by the user and the preset planning rules.

[0574] In some embodiments, the processor is further configured to: determine the target work area in the displayed image.

[0575] In some embodiments, the step of determining the target operating area includes at least one of the following:

[0576] Get the user's selection operation on the target operation area; or

[0577] The target operating area is automatically determined according to preset target operating area determination conditions.

[0578] In some embodiments, the path planning method is applied to a control terminal, the control terminal is configured to be communicatively connected with the movable platform, and the control terminal is configured to be able to display the real-time view picture.

[0579] In some embodiments, the processor is further configured to implement:

[0580] The movable platform is controlled to operate according to the operation path.

[0581] In some embodiments, the movable platform includes an aerial photography aircraft, a ground viewing device, a mobile scanning device, a mobile measuring device, a mobile mapping device, a camera, or a mobile phone.

[0582] In some embodiments, the target operation area is a planar operation area.

[0583] In some embodiments, the target operation area includes at least one of the following:

[0584] Vertical working area, inclined working area or flat working area.

[0585] In some embodiments, the real-time view image displays the distance between the target working area and the movable platform.

[0586] In some embodiments, the processor is further configured to implement:

[0587] When the distance between the target operation area and the movable platform exceeds a preset distance range, a prompt message is issued.

[0588] In some embodiments, the processor is further configured to implement:

[0589] Determining whether there is a path segment in the operation path that collides with an obstacle;

[0590] If there is a path segment that collides with an obstacle in the operation path, the path segment that collides with the obstacle is marked in the operation path.

[0591] In some other embodiments, the processor is configured to run a computer program stored in the memory, and implement the following steps when executing the computer program:

[0592] Display the three-dimensional shape of the target object in the image;

[0593] In response to a target surface determination operation, determining a target surface of the target object from the three-dimensional shape of the target object;

[0594] Automatically generate a reference surface based on the target surface; and

[0595] Based on the reference surface, a working path of the movable platform is generated.

[0596] In some embodiments, the three-dimensional shape of the target object includes a surface of the target object, and the target surface is related to the surface of the target object.

[0597] In some embodiments, in response to the target plane determination operation, determining the target plane of the target object from the three-dimensional shape of the target object includes:

[0598] In response to a user's selection operation on one of the surfaces of the three-dimensional shape, the selected surface on the three-dimensional shape of the target object is selected as the target surface.

[0599] In some embodiments, the reference surface is substantially parallel to at least a portion of the target surface.

[0600] In some embodiments, automatically generating a reference surface based on the target surface includes:

[0601] Determining the geographic location information of the reference surface based on the geographic location information of the movable platform and the relative positional relationship between the target surface and the movable platform, wherein the image is a real-time view of the movable platform; and

[0602] The reference surface is automatically generated based on the geographical location information of the reference surface.

[0603] In some embodiments, the relative position relationship between the target surface and the movable platform is determined by information obtained by a ranging sensor.

[0604] In some embodiments, automatically generating a reference surface based on the target surface includes:

[0605] Determine the geographic location information of the target surface based on the pre-generated three-mode model; and

[0606] The reference surface is automatically generated based on the geographic location information of the target surface.

[0607] In some embodiments, the processor is further configured to implement:

[0608] Adjusting the reference surface based on a positional relationship between the reference surface and the target object in the image so that the adjusted reference surface coincides with the target surface or a deviation between the adjusted reference surface and the target surface is less than a preset value; and

[0609] A target operating path of the movable platform is generated based on the adjusted reference surface.

[0610] In some embodiments, the processor is further configured to implement:

[0611] In response to a user's adjustment operation on the reference surface, adjusting the reference surface so that the adjusted reference surface coincides with the target surface or a deviation between the adjusted reference surface and the target surface is less than a preset value;

[0612] as well as

[0613] Based on the adjusted reference surface, an operation path of the movable platform is generated.

[0614] In some embodiments, adjusting the reference plane includes:

[0615] The geometric information of the reference surface is adjusted.

[0616] In some embodiments, the geometric information of the reference surface includes at least one of the following: the area of ​​the reference surface, the shape of the reference surface, and the tilt angle of the reference surface.

[0617] In some embodiments, at least a portion of the working path is generated on the reference surface.

[0618] In some embodiments, at least a portion of the working path is substantially parallel to the reference surface.

[0619] In some embodiments, the method further comprises:

[0620] Adjusting the operation path based on the positional relationship between the operation path and each object in the image, and displaying the adjusted operation path in the image; or

[0621] In response to a user's adjustment operation on the work path, the work path is adjusted, and the adjusted work path is displayed in the image.

[0622] In some embodiments, the processor is further configured to implement:

[0623] Based on the positional relationship between the operation path and each object in the image, a path segment with a collision risk in the operation path is determined, and the path segment with a collision risk is marked.

[0624] In some embodiments, the processor is further configured to implement:

[0625] Changing the image, where the changed image has a different viewing angle than the image before the change;

[0626] In the changed image, the reference surface continues to be displayed.

[0627] In some embodiments, the image is a real image, and the reference surface is a virtual image superimposed on the real image.

[0628] In some embodiments, the image and the reference surface are virtual images.

[0629] In some embodiments, the image is a 2D image or a 3D image acquired based on an image sensor; or, the image is a 2D image or a 3D image generated based on sensing information of a ranging sensor.

[0630] In some embodiments, the image sensor and / or the ranging sensor is mounted on the movable platform.

[0631] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the path planning device described above can refer to the corresponding process in the aforementioned path planning method embodiment, and will not be repeated here.

[0632] Please refer to Figure 25, which is a schematic block diagram of the structure of a control device provided in an embodiment of the present application. The control device is applied to a control terminal and can also be applied to a mobile platform.

[0633] As shown in FIG. 25 , the control device 400 includes a processor 410 and a memory 420 . The processor 410 and the memory 420 are connected via a bus 430 , which is, for example, an I 2 C (Inter-Integrated Circuit) bus.

[0634] Specifically, the processor 410 may be a microcontroller unit (MCU), a central processing unit (CPU), or a digital signal processor (DSP).

[0635] Specifically, the memory 420 may be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a mobile hard disk.

[0636] The processor 410 is configured to run a computer program stored in the memory 420 and implement the following steps when executing the computer program:

[0637] In response to a user's operation, controlling the movable platform to move to the vicinity of a target object, wherein the movable platform is equipped with an image sensor and a ranging sensor;

[0638] displaying an image of a target surface including the target object acquired by the image sensor;

[0639] determining a reference surface in the image based at least on the sensing information of the ranging sensor, wherein position information of the reference surface is associated with position information of the target surface;

[0640] In response to a user's adjustment operation on the reference surface, adjusting the reference surface in the image to obtain an adjusted reference surface;

[0641] generating an operating path of the movable platform based on the adjusted reference surface; and

[0642] The movable platform is controlled to move along the working path.

[0643] In some embodiments, the step of adjusting the reference surface in the image in response to a user's adjustment operation on the reference surface includes:

[0644] In response to a user's adjustment operation on the reference surface, geometric information of the reference surface is adjusted in the image.

[0645] In some embodiments, the geometric information of the reference surface includes at least one of the following: the area of ​​the reference surface, the shape of the reference surface, and the tilt angle of the reference surface.

[0646] In some embodiments, the reference surface is substantially parallel to at least a portion of the target surface.

[0647] In some embodiments, at least part of the working path is generated on the reference surface; or,

[0648] At least a portion of the working path is substantially parallel to the reference surface.

[0649] In some embodiments, the processor is further configured to implement:

[0650] Adjusting the operation path based on the positional relationship between the operation path and each object in the image, and displaying the adjusted operation path in the image; or

[0651] In response to a user's adjustment operation on the work path, the work path is adjusted, and the adjusted work path is displayed in the image.

[0652] In some embodiments, the processor is further configured to implement:

[0653] Based on the positional relationship between the operation path and each object in the image, a path segment with a collision risk in the operation path is determined, and the path segment with a collision risk is marked.

[0654] In some embodiments, the processor is further configured to implement:

[0655] Changing the image, where the changed image has a different viewing angle than the image before the change;

[0656] In the changed image, the reference surface continues to be displayed.

[0657] In some embodiments, determining a reference surface in the image based at least on the sensing information of the ranging sensor includes:

[0658] determining the distance between the target surface and the movable platform based on information acquired by the ranging sensor; and

[0659] The geographic location information of the reference surface is determined based on the distance between the target surface and the movable platform.

[0660] In some embodiments, the geographic location information of the reference surface is determined based on a ranging direction for determining a distance between the target surface and the movable platform.

[0661] In some embodiments, the geographic location information of the reference surface is determined by information acquired by the ranging sensor and the geographic location information of the movable platform.

[0662] In some embodiments, the image is a real image, and the reference surface is a virtual image superimposed on the real image.

[0663] In some embodiments, the image and the reference surface are virtual images.

[0664] It should be noted that, those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the control device of the movable platform described above can refer to the corresponding process in the aforementioned embodiment of the control method of the movable platform, and will not be repeated here.

[0665] Please refer to Figure 26, which is a schematic block diagram of the structure of a control terminal provided in an embodiment of the present application.

[0666] As shown in FIG26 , the control terminal 200 includes a processor 201 and a memory 202 . The processor 201 and the memory 202 are connected via a bus 203 , which is, for example, an I 2 C (Inter-Integrated Circuit) bus.

[0667] Specifically, the processor 201 may be a microcontroller unit (MCU), a central processing unit (CPU), or a digital signal processor (DSP).

[0668] Specifically, the memory 202 may be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a mobile hard disk.

[0669] The processor 201 is configured to run a computer program stored in the memory 202 and implement the steps of the path planning method or control method provided in the above embodiments when executing the computer program.

[0670] In some embodiments, the processor is configured to implement the following steps:

[0671] Determine the target operating area;

[0672] Automatically generating a virtual work area on a real-time view screen of a mobile platform according to preset conditions, wherein geographical location information corresponding to the virtual work area in real three-dimensional space is determined based on the preset conditions;

[0673] Adjusting the virtual operating area according to the target operating area; and

[0674] Plan the operation path based on the adjusted virtual operation area.

[0675] In some other embodiments, the processor is configured to run a computer program stored in the memory, and implement the following steps when executing the computer program:

[0676] Determine the target operating area;

[0677] determining a distance between the target operating area and the movable platform;

[0678] generating a virtual work area on a real-time viewing screen of the movable platform based on parameters related to the distance; and

[0679] Based on the virtual operation area, an operation path is planned.

[0680] In some other embodiments, the processor is configured to run a computer program stored in the memory, and implement the following steps when executing the computer program:

[0681] Display the three-dimensional shape of the target object in the image;

[0682] In response to a target surface determination operation, determining a target surface of the target object from the three-dimensional shape of the target object;

[0683] Automatically generate a reference surface based on the target surface; and

[0684] Based on the reference surface, a working path of the movable platform is generated.

[0685] In some other embodiments, the processor is configured to run a computer program stored in the memory, and implement the following steps when executing the computer program:

[0686] In response to a user's operation, controlling the movable platform to move to the vicinity of a target object, wherein the movable platform is equipped with an image sensor and a ranging sensor;

[0687] displaying an image of a target surface including the target object acquired by the image sensor;

[0688] determining a reference surface in the image based at least on the sensing information of the ranging sensor, wherein position information of the reference surface is associated with position information of the target surface;

[0689] In response to a user's adjustment operation on the reference surface, adjusting the reference surface in the image to obtain an adjusted reference surface;

[0690] generating an operating path of the movable platform based on the adjusted reference surface; and

[0691] The movable platform is controlled to move along the working path.

[0692] It should be noted that technical personnel in the relevant field can clearly understand that for the convenience and conciseness of description, the specific working process of the control terminal described above can refer to the corresponding process in the aforementioned path planning method or control method embodiment, and will not be repeated here.

[0693] An embodiment of the present application also provides a storage medium for computer-readable storage, wherein the storage medium stores a computer program, wherein the computer program includes program instructions, and the processor executes the program instructions to implement the steps of the path planning method or control method provided in the above embodiment.

[0694] The storage medium may be an internal storage unit of the control terminal or mobile platform described in any of the aforementioned embodiments, such as a hard disk or memory of the control terminal or mobile platform. The storage medium may also be an external storage device of the control terminal or mobile platform, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the control terminal or mobile platform.

[0695] It should be understood that the terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0696] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0697] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present application, and such modifications or substitutions should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A path planning method, characterized in that: The method comprises: Determine the target operating area; Automatically generating a virtual work area on a real-time view screen of a mobile platform according to preset conditions, wherein geographical location information corresponding to the virtual work area in real three-dimensional space is determined based on the preset conditions; Adjusting the virtual operating area according to the target operating area; and Plan the operation path based on the adjusted virtual operation area.

2. The path planning method according to claim 1, characterized in that: The preset condition includes: determining the position of the virtual working area based on the relative position relationship between the target working area and the movable platform.

3. The path planning method according to claim 2, characterized in that: The determining the position of the virtual operating area based on the relative positional relationship between the target operating area and the movable platform includes: The position of the virtual work area is determined based on the distance between the target work area and the movable platform.

4. The path planning method according to claim 3, characterized in that: The distance between the position corresponding to the virtual work area in the real three-dimensional space and the movable platform is equal to or approximately equal to the distance between the target work area and the movable platform; or, there is a preset difference between the distance between the position corresponding to the virtual work area in the real three-dimensional space and the movable platform and the distance between the target work area and the movable platform.

5. The path planning method according to claim 3 or 4, characterized in that: The distance between the movable platform and the target operating area is measured by a distance measuring sensor; or the distance between the target operating area and the movable platform is determined based on a pre-generated perception map and the geographical location information of the movable platform itself.

6. The path planning method according to claim 2 or 3, characterized in that: The determining the position of the virtual operating area based on the relative positional relationship between the target operating area and the movable platform includes: The orientation of the virtual work area is determined based on a distance direction, where the distance direction is the direction along which the distance between the movable platform and the target work area is determined; or, the orientation of the virtual work area is determined based on a viewing direction of the movable platform.

7. The path planning method according to claim 6, characterized in that: When the orientation of the virtual work area is determined based on the fixed distance direction, the projection direction of the virtual work area is consistent with the fixed distance direction, or an angle is set between the projection direction of the virtual work area and the fixed distance direction; when the orientation of the virtual work area is determined based on the framing direction of the movable platform, the projection direction of the virtual work area is consistent with the framing direction, or an angle is set between the projection direction of the virtual work area and the framing direction.

8. The path planning method according to claim 6 or 7, characterized in that: The viewing direction of the movable platform is consistent with the distance direction.

9. The path planning method according to claim 6 or 7, characterized in that: The viewing direction of the movable platform is perpendicular or approximately perpendicular to the target operating area, so that the generated virtual operating area is parallel or approximately parallel to the target operating area.

10. The path planning method according to claim 9, characterized in that: Before the step of determining the target operating area, the method further includes: Obtaining an adjustment instruction from a user, where the adjustment instruction is used to adjust the viewing angle of the movable platform; and According to the adjustment instruction, the viewing angle of the movable platform is adjusted so that the viewing angle of the movable platform is perpendicular or approximately perpendicular to the target operating area; or determining the angle of the target operating area based on a pre-generated perception map; and Based on the angle of the target working area, the viewing angle of the movable platform is automatically determined so that the viewing direction of the movable platform is perpendicular or approximately perpendicular to the target working area.

11. The path planning method according to claim 2, characterized in that: The determining the position of the virtual operating area based on the relative positional relationship between the target operating area and the movable platform includes: The position of the virtual work area is determined based on the orientation of the target work area relative to the movable platform.

12. The path planning method according to claim 1, characterized in that: Also includes: In response to the target operation area being determined, a virtual operation area is automatically generated on the real-time view screen of the movable platform according to preset conditions.

13. The path planning method according to claim 1 or 12, characterized in that: Before the step of determining the target operating area, a preselection box is displayed on the live view screen, and the preselection box is used to indicate that the area selected on the live view screen is the target operating area; The step of automatically generating a virtual operating area on the real-time viewing screen of the movable platform according to preset conditions includes: The virtual operation area is automatically generated in the live view image based on the pre-selected box and the preset conditions.

14. The path planning method according to claim 13, characterized in that: The step of generating the virtual operation area in the live view image based on the pre-selected box and the preset conditions includes: The size of the virtual operation area in the live view image is determined based on the pre-selected frame, and the position of the virtual operation area is determined based on the preset condition.

15. The path planning method according to claim 13, wherein: The size of the pre-selection frame in the live view image is a preset size.

16. The path planning method according to claim 1 or 12, characterized in that: The step of automatically generating a virtual operating area on the real-time viewing screen of the movable platform according to preset conditions includes: Extracting a contour line of the target operating area in the real-time view image according to the determined target operating area; and The virtual operation area is automatically generated in the live view image based on the area defined by the contour line and the preset conditions.

17. The path planning method according to claim 16, characterized in that: The step of generating the virtual working area in the live view screen based on the area defined by the contour line and the preset conditions includes: The size of the virtual work area in the live view image is determined based on the area delimited by the contour line, and the position of the virtual work area is determined based on the preset condition.

18. The path planning method according to claim 12, wherein: The step of automatically generating a virtual operating area on the real-time viewing screen of the movable platform according to preset conditions includes: Determining a plurality of feature points according to the determined target operation area; Obtaining a relative positional relationship between each of the feature points and the movable platform; Determining the geographical location information of each of the feature points based on the geographical location information of the movable platform and the relative position relationship; and The virtual operation area is automatically generated based on the geographical location information of each feature point.

19. The path planning method according to claim 18, characterized in that: The step of generating the virtual operating area based on the geographical location information of each feature point includes: According to the geographical location information of the feature point, a point mark is superimposed on the corresponding feature point; The plurality of point marks are connected according to the actual relative positional relationship between the plurality of feature points, thereby generating and displaying the virtual operation area covering the target operation area.

20. The path planning method according to claim 18, wherein: The feature points include at least endpoints of a boundary line of the target operation area.

21. The path planning method according to claim 18, characterized in that: The relative position relationship includes the distance between the feature point and the movable platform, and a measurement direction when measuring the distance between the feature point and the movable platform.

22. The path planning method according to claim 18, wherein: The step of determining a plurality of feature points includes: Obtain the user's operation of determining multiple feature points; or, Automatically extract multiple feature points.

23. The path planning method according to claim 1, characterized in that: The step of adjusting the virtual operating area according to the target operating area includes: adjusting the virtual operating area according to the target operating area so that the adjusted virtual operating area coincides with the target operating area or the deviation between the adjusted virtual operating area and the target operating area is less than a preset value.

24. The path planning method according to claim 23, characterized in that: In the step of adjusting the virtual operating area according to the target operating area, the user visually determines whether the virtual operating area and the target operating area overlap or whether the deviation between the virtual operating area and the target operating area is less than the preset value.

25. The path planning method according to claim 23 or 24, characterized in that: The deviation between the virtual operating area and the target operating area is less than a preset value, including at least one of the following: The distance between the virtual work area and the target work area in the Z-axis direction of the preset three-dimensional coordinate system is less than a first preset value, the plane formed by the X-axis and the Y-axis of the preset three-dimensional coordinate system is parallel to the virtual work area, and the Z-axis is perpendicular to the virtual work area; The tilt angle between the virtual operating area and the target operating area is less than a second preset value, the tilt angle being the angle at which the virtual operating area is tilted relative to the target operating area around the X-axis or the Y-axis; The distance between the outline of the virtual operating area and the outline of the target operating area is less than a third preset value; A deviation angle between the virtual operating area and the target operating area in the Z-axis direction is smaller than a fourth preset value.

26. The path planning method according to claim 1, 23 or 24, characterized in that: The step of adjusting the virtual operating area according to the target operating area includes at least a first operation and / or a second operation; wherein, The first operation is used to adjust at least one of the size, shape, position and orientation of the virtual work area; The second operation is used to adjust the distance between the virtual working area and the target working area and / or the angle of the virtual working area relative to the target working area.

27. The path planning method according to claim 1, characterized in that: The step of adjusting the virtual operating area according to the target operating area includes: The movable platform is adjusted so that the viewing direction of the movable platform is switched to another angle of the target operation area, and the real-time viewing screen presents the target operation area and the virtual operation area at the other angle, wherein: The other angle is different from the viewing angle of the movable platform when generating the virtual work area; The virtual operating area is adjusted according to the target operating area at the other angle and the virtual operating area.

28. The path planning method according to claim 27, characterized in that: The target operation area is a planar area, and adjusting the movable platform so that the viewing direction of the movable platform is switched to another angle of the target operation area includes: Adjusting the movable platform so that the viewing direction of the movable platform is switched to the side of the target working area, and the real-time viewing image includes the side of the target working area and the side of the virtual working area; The adjusting the virtual operating area according to the target operating area at the other angle and the virtual operating area to obtain an adjusted virtual operating area includes: When the viewing direction of the movable platform is switched to the side of the target working area, the distance between the virtual working area and the target working area is adjusted and / or the angle of the virtual working area relative to the target working area is adjusted.

29. The path planning method according to claim 28, characterized in that: Before determining the target operating area, the method further includes: The movable platform is adjusted so that the viewing direction of the movable platform is the front of the target operation area, and the real-time viewing picture at least includes the front part of the target operation area.

30. The path planning method according to claim 29, characterized in that: The step of adjusting the virtual working area according to the target working area includes adjusting at least one of the size, shape, position and orientation of the virtual working area according to the target working area in front.

31. The path planning method according to claim 1 or 23, characterized in that: The step of adjusting the virtual operating area according to the target operating area includes: adjusting the virtual operation area based on an acquired area adjustment operation of the user, wherein the user adjusts the virtual operation area according to the target operation area; or The virtual operating area is automatically adjusted according to the target operating area and a preset area adjustment algorithm.

32. The path planning method according to claim 31, characterized in that: The step of obtaining the user's region adjustment operation includes at least one of the following: Acquiring the area adjustment operation in the live view picture; or The area adjustment operation is obtained in an overview image, wherein the overview image displays the target operation area and the virtual operation area viewed by the movable platform at a preset angle.

33. The path planning method according to claim 32, characterized in that: When the area adjustment operation is acquired in the live view image, the virtual work area in the overview image also changes in response to the area adjustment operation; and / or, When the area adjustment operation is acquired in the overview image, the virtual work area in the live view screen also changes in response to the area adjustment operation.

34. The path planning method according to claim 32 or 33, characterized in that: The method further comprises: The overview map is automatically generated in response to the generation of the virtual work area; or, the overview map is obtained in response to a user's photographing operation.

35. The path planning method according to claim 31, characterized in that: The virtual operation area includes at least one adjustment point located on the boundary of the virtual operation area, and the area adjustment operation includes a user dragging or clicking operation on the adjustment point; and / or, the virtual operation area includes multiple endpoint addition buttons and / or endpoint deletion buttons, and the area adjustment operation includes a user dragging or clicking operation on the endpoint addition button and / or the endpoint deletion button.

36. The path planning method according to claim 1, characterized in that: The path planning method is applied to a control terminal, the control terminal is configured to be communicatively connected with the movable platform, and the control terminal is configured to be able to display the real-time view picture.

37. The path planning method according to claim 1 or 36, characterized in that: The method further comprises: The movable platform is controlled to operate according to the operation path.

38. The path planning method according to claim 1, characterized in that: The step of planning the operation path based on the adjusted virtual operation area includes: generating the operation path based on the adjusted virtual operation area; or An initial operation path is generated based on the virtual operation area, and based on the adjustment of the virtual operation area, the initial operation path is adaptively adjusted, thereby obtaining the operation path planned based on the adjusted virtual operation area.

39. The path planning method according to claim 1 or 38, characterized in that: The step of planning the operation path includes: The "bow" or "Z"-shaped operation path is automatically generated based on the virtual operation area, the operation parameters set by the user, and the preset planning rules.

40. The path planning method according to claim 1, characterized in that: The step of adjusting the virtual operating area according to the target operating area further includes: Based on the obtained geographical location information of the virtual operating area and the amount of change in the geographical location information corresponding to the adjustment of the virtual operating area, the adjusted geographical location information of the virtual operating area is determined.

41. The path planning method according to claim 40, characterized in that: The method further comprises: The geographical location information of the virtual work area is determined based on the geographical location information of the movable platform and the preset condition.

42. The path planning method according to claim 40, characterized in that: The amount of change in the geographical location information corresponding to the adjustment of the virtual operating area is determined according to a preset rule.

43. The path planning method according to claim 1, characterized in that: The target work area is determined in the displayed image.

44. The path planning method according to claim 1 or 43, characterized in that: The step of determining the target operating area includes at least one of the following: Get the user's selection operation on the target operation area; or The target operating area is automatically determined according to preset target operating area determination conditions.

45. The path planning method according to claim 1, characterized in that: The movable platform includes an aerial photography aircraft, a ground viewing device, a mobile scanning device, a mobile measuring device, a mobile mapping device, a camera or a mobile phone.

46. ​​The path planning method according to claim 1, characterized in that: The target operation area is a planar operation area.

47. The path planning method according to claim 46, characterized in that: The target operation area includes at least one of the following: Vertical working area, inclined working area or flat working area.

48. The path planning method according to claim 1, characterized in that: The method further comprises: Determining whether there is a path segment in the operation path that collides with an obstacle; If there is a path segment that collides with an obstacle in the operation path, the path segment that collides with the obstacle is marked in the operation path.

49. A path planning method, characterized in that: include: Determine the target operating area; determining a distance between the target operating area and the movable platform; generating a virtual work area on a real-time viewing screen of the movable platform based on parameters related to the distance; as well as Based on the virtual operation area, an operation path is planned.

50. The path planning method according to claim 49, characterized in that: The virtual operation area is superimposed on the target operation area.

51. The path planning method according to claim 49, characterized in that: The parameters related to the distance include: the distance between the target working area and the movable platform and / or a distance direction, wherein the distance between the target working area and the movable platform is determined along the distance direction.

52. The path planning method according to any one of claims 49 to 51, characterized in that: The step of generating a virtual operating area on the real-time view screen of the movable platform based on the parameters related to the distance includes: Based on the distance between the target work area and the movable platform, the distance between the virtual work area and the movable platform is determined.

53. The path planning method according to claim 52, characterized in that: The distance between the position corresponding to the virtual work area in the real three-dimensional space and the movable platform is equal to or approximately equal to the distance between the target work area and the movable platform; or, there is a preset difference between the distance between the position corresponding to the virtual work area in the real three-dimensional space and the movable platform and the distance between the target work area and the movable platform.

54. The path planning method according to any one of claims 49 to 51, characterized in that: The distance between the movable platform and the target operating area is measured by a distance measuring sensor; or the distance between the target operating area and the movable platform is determined based on a pre-generated perception map and the geographical location information of the movable platform itself.

55. The path planning method according to claim 52, characterized in that: The step of generating a virtual working area on the real-time view screen of the movable platform based on the parameters related to the distance further includes: The orientation of the virtual work area is determined based on the distance direction, or the orientation of the virtual work area is determined based on the viewing direction of the movable platform.

56. The path planning method according to claim 55, characterized in that: When the orientation of the virtual work area is determined based on the fixed distance direction, the projection direction of the virtual work area is consistent with the fixed distance direction, or an angle is set between the projection direction of the virtual work area and the fixed distance direction; when the orientation of the virtual work area is determined based on the framing direction of the movable platform, the projection direction of the virtual work area is consistent with the framing direction, or an angle is set between the projection direction of the virtual work area and the framing direction.

57. The path planning method according to claim 55, characterized in that: The viewing direction of the movable platform is consistent with the distance direction.

58. The path planning method according to any one of claims 55 to 57, characterized in that: The viewing direction of the movable platform is perpendicular or approximately perpendicular to the target operating area, so that the generated virtual operating area is parallel or approximately parallel to the target operating area.

59. The path planning method according to claim 49, characterized in that: Before the step of determining the target operating area, the method further includes: Obtaining an adjustment instruction from a user, where the adjustment instruction is used to adjust the viewing angle of the movable platform; and According to the adjustment instruction, the viewing angle of the movable platform is adjusted so that the viewing direction of the movable platform is perpendicular or approximately perpendicular to the target operation area; or, determining the angle of the target operating area based on a pre-generated perception map; and Based on the angle of the target working area, the viewing angle of the movable platform is automatically determined so that the viewing direction of the movable platform is perpendicular or approximately perpendicular to the target working area.

60. The path planning method according to claim 49, characterized in that: Before the step of determining the target operating area, a preselection box is displayed on the live view screen, and the preselection box is used to indicate that the area selected on the live view screen is the target operating area; The step of generating a virtual operating area on the real-time view screen of the movable platform based on the parameters related to the distance includes: The virtual work area is generated in the live view screen based on the parameters related to the distance and the pre-selected box.

61. The path planning method according to claim 60, characterized in that: The step of generating the virtual operating area in the real-time view image based on the parameters related to the distance and the pre-selected box includes: The size of the virtual work area in the live view image is determined based on the pre-selected box, and the position corresponding to the virtual work area in the real three-dimensional space is determined based on parameters related to the distance.

62. The path planning method according to claim 61, characterized in that: Before the step of determining the target operating area, the method further includes: Displaying the pre-selected frame on the live view screen; as well as A framing adjustment operation is acquired, and according to the framing adjustment operation, a framing direction or size is adjusted so that the pre-selected frame at least partially covers the target operation area.

63. The path planning method according to claim 49, characterized in that: The step of generating a virtual operating area on the real-time view screen of the movable platform based on the parameters related to the distance includes: Extracting a contour line of the target operating area in the real-time view image according to the determined target operating area; and The virtual work area is generated in the live view screen based on the area demarcated by the contour line and the determined parameters related to the distance.

64. The path planning method according to claim 63, characterized in that: The step of generating the virtual working area in the live view screen based on the area defined by the contour line and the determined parameters related to the distance includes: The size of the virtual work area in the live view image is determined based on the area delimited by the contour line, and the position corresponding to the virtual work area in the real three-dimensional space is determined based on parameters related to the distance.

65. The path planning method according to claim 49, characterized in that: The steps of determining the distance between the target operating area and the movable platform, and generating a virtual operating area in a real-time view image of the movable platform based on parameters related to the distance, include: Determining a plurality of feature points according to the determined target operation area; Obtaining the distance between each of the feature points and the movable platform; Determining the geographical location information of each of the feature points based on the geographical location information of the movable platform and the parameter related to the distance; and The virtual operation area is generated based on the geographical location information of each feature point.

66. The path planning method according to claim 65, characterized in that: The step of generating the virtual operating area based on the geographical location information of each feature point includes: According to the geographical location information of the feature point, a point mark is superimposed on the corresponding feature point; The plurality of point marks are connected according to the actual relative positional relationship between the plurality of feature points, thereby generating and displaying the virtual operation area covering the target operation area.

67. The path planning method according to claim 65, characterized in that: The feature points include at least endpoints of a boundary line of the target operation area.

68. The path planning method according to claim 65, characterized in that: The parameters related to the distance include the distance between the feature point and the movable platform, and a measurement direction when measuring the distance between the feature point and the movable platform.

69. The path planning method according to claim 65, characterized in that: The step of determining a plurality of feature points includes: Obtain the user's operation of determining multiple feature points; or, Automatically extract multiple feature points.

70. The path planning method according to claim 49, characterized in that: The method further comprises: Adjusting the virtual operating area according to the target operating area; and The operation path is planned based on the adjusted virtual operation area.

71. The path planning method according to claim 70, characterized in that: The step of adjusting the virtual operating area according to the target operating area includes: adjusting the virtual operating area according to the target operating area so that the adjusted virtual operating area coincides with the target operating area or the deviation between the adjusted virtual operating area and the target operating area is less than a preset value.

72. The path planning method according to claim 71, characterized in that: In the step of adjusting the virtual operating area according to the target operating area, the user visually determines whether the virtual operating area and the target operating area overlap or whether the deviation between the virtual operating area and the target operating area is less than the preset value.

73. The path planning method according to claim 71 or 72, characterized in that: The deviation between the virtual operating area and the target operating area is less than a preset value, including at least one of the following: The distance between the virtual work area and the target work area in the Z-axis direction of the preset three-dimensional coordinate system is less than a first preset value, the plane formed by the X-axis and the Y-axis of the preset three-dimensional coordinate system is parallel to the virtual work area, and the Z-axis is perpendicular to the virtual work area; The tilt angle between the virtual operating area and the target operating area is less than a second preset value, the tilt angle being the angle at which the virtual operating area is tilted relative to the target operating area around the X-axis or the Y-axis; The distance between the outline of the virtual operating area and the outline of the target operating area is less than a third preset value; A deviation angle between the virtual operating area and the target operating area in the Z-axis direction is smaller than a fourth preset value.

74. The path planning method according to claim 70, 71 or 72, wherein: The step of adjusting the virtual operating area according to the target operating area includes at least a first operation and / or a second operation; wherein, The first operation is used to adjust at least one of the size, shape, position and orientation of the virtual work area; The second operation is used to adjust the distance between the virtual working area and the target working area and / or the angle of the virtual working area relative to the target working area.

75. The path planning method according to claim 70, characterized in that: The step of adjusting the virtual operating area according to the target operating area includes: Adjusting the movable platform so that the viewing direction of the movable platform is switched to another angle of the target work area, and the real-time viewing screen presents the target work area and the virtual work area at the other angle, wherein the other angle is different from the viewing angle of the movable platform when generating the virtual work area; The virtual operating area is adjusted according to the target operating area at the other angle and the virtual operating area.

76. The path planning method according to claim 75, characterized in that: The target operation area is a planar area, and adjusting the movable platform so that the viewing direction of the movable platform is switched to another angle of the target operation area includes: Adjusting the movable platform so that the viewing direction of the movable platform is switched to the side of the target working area, and the real-time viewing image includes the side of the target working area and the side of the virtual working area; The adjusting the virtual operating area according to the target operating area at the other angle and the virtual operating area to obtain an adjusted virtual operating area includes: When the viewing direction of the movable platform is switched to the side of the target working area, the distance between the virtual working area and the target working area is adjusted and / or the angle of the virtual working area relative to the target working area is adjusted.

77. The path planning method according to claim 76, characterized in that: Before determining the target operating area, the method further includes: The movable platform is adjusted so that the viewing direction of the movable platform is the front of the target operation area, and the real-time viewing picture at least includes the front part of the target operation area.

78. The path planning method according to claim 77, characterized in that: The step of adjusting the virtual working area according to the target working area includes adjusting at least one of the size, shape, position and orientation of the virtual working area according to the target working area in front.

79. The path planning method according to claim 70, characterized in that: The step of adjusting the virtual operating area according to the target operating area includes: adjusting the virtual operation area based on an acquired area adjustment operation of the user, wherein the user adjusts the virtual operation area according to the target operation area; or The virtual operating area is automatically adjusted according to the target operating area and a preset area adjustment algorithm.

80. The path planning method according to claim 79, characterized in that: The step of obtaining the user's region adjustment operation includes at least one of the following: Acquiring the area adjustment operation in the live view picture; or The area adjustment operation is obtained in an overview image, wherein the overview image displays the target operation area and the virtual operation area viewed by the movable platform at a preset angle.

81. The path planning method according to claim 80, characterized in that: When the area adjustment operation is acquired in the live view image, the virtual work area in the overview image also changes in response to the area adjustment operation; and / or, When the area adjustment operation is acquired in the overview image, the virtual work area in the live view screen also changes in response to the area adjustment operation.

82. The path planning method according to claim 80 or 81, characterized in that: The method further comprises: The overview map is automatically generated in response to the generation of the virtual work area; or, the overview map is obtained in response to a user's photographing operation.

83. The path planning method according to claim 70, characterized in that: The virtual operation area includes at least one adjustment point located on the boundary of the virtual operation area, and the area adjustment operation includes a user dragging or clicking operation on the adjustment point; and / or, the virtual operation area includes multiple endpoint addition buttons and / or endpoint deletion buttons, and the area adjustment operation includes a user dragging or clicking operation on the endpoint addition button and / or the endpoint deletion button.

84. The path planning method according to claim 70, characterized in that: A perception map is also superimposed and displayed on the real-time view screen. The perception map includes a perception model of the target operation area. The perception model is used to provide a reference basis for the area adjustment operation.

85. The path planning method according to claim 70, characterized in that: The step of adjusting the virtual operating area according to the target operating area further includes: Based on the obtained geographical location information of the virtual operating area and the amount of change in the geographical location information corresponding to the adjustment of the virtual operating area, the adjusted geographical location information of the virtual operating area is determined.

86. The path planning method according to claim 85, characterized in that: The method further comprises: The geographical location information of the virtual work area is determined based on the geographical location information of the movable platform and a parameter related to the distance.

87. The path planning method according to claim 85, characterized in that: The amount of change in the geographical location information corresponding to the adjustment of the virtual operating area is determined according to a preset rule.

88. The path planning method according to claim 70, characterized in that: The step of planning an operation path based on the virtual operation area includes: generating the operation path based on the adjusted virtual operation area; or An initial operation path is generated based on the virtual operation area, and based on the adjustment of the virtual operation area, the initial operation path is adaptively adjusted, thereby obtaining the operation path planned based on the adjusted virtual operation area.

89. The path planning method according to claim 49 or 88, characterized in that: The step of planning the operation path includes: The "bow" or "Z"-shaped operation path is automatically generated based on the virtual operation area, the operation parameters set by the user, and the preset planning rules.

90. The path planning method according to claim 49, characterized in that: The target working area is determined in the displayed image.

91. The path planning method according to claim 49 or 90, characterized in that: The step of determining the target operating area includes at least one of the following: Get the user's selection operation on the target operation area; or The target operating area is automatically determined according to preset target operating area determination conditions.

92. The path planning method according to claim 49, characterized in that: The path planning method is applied to a control terminal, the control terminal is configured to be communicatively connected with the movable platform, and the control terminal is configured to be able to display the real-time view picture.

93. The path planning method according to claim 49 or 92, characterized in that: The method further comprises: The movable platform is controlled to operate according to the operation path.

94. The path planning method according to claim 49, characterized in that: The movable platform includes an aerial photography aircraft, a ground viewing device, a mobile scanning device, a mobile measuring device, a mobile mapping device, a camera or a mobile phone.

95. The path planning method according to claim 49, characterized in that: The target operation area is a planar operation area.

96. The path planning method according to claim 95, characterized in that: The target operation area includes at least one of the following: Vertical working area, inclined working area or flat working area.

97. The path planning method according to claim 49, characterized in that: The method further comprises: Determining whether there is a path segment in the operation path that collides with an obstacle; If there is a path segment that collides with an obstacle in the operation path, the path segment that collides with the obstacle is marked in the operation path.

98. A path planning method for a movable platform, characterized in that: include: Display the three-dimensional shape of the target object in the image; In response to a target surface determination operation, determining a target surface of the target object from the three-dimensional shape of the target object; Automatically generate a reference surface based on the target surface; as well as Based on the reference surface, a working path of the movable platform is generated.

99. The path planning method according to claim 98, characterized in that: The three-dimensional shape of the target object includes a surface of the target object, and the target surface is related to the surface of the target object.

100. The path planning method according to claim 99, characterized in that: The step of determining the target surface of the target object from the three-dimensional shape of the target object in response to the target surface determination operation includes: In response to a user's selection operation on one of the surfaces of the three-dimensional shape, the selected surface on the three-dimensional shape of the target object is selected as the target surface.

101. The path planning method according to any one of claims 98 to 100, characterized in that: The reference surface is substantially parallel to at least a portion of the target surface.

102. The path planning method according to claim 98, characterized in that: The automatically generating a reference surface based on the target surface includes: Determining the geographic location information of the reference surface based on the geographic location information of the movable platform and the relative positional relationship between the target surface and the movable platform, wherein the image is a real-time view of the movable platform; and The reference surface is automatically generated based on the geographical location information of the reference surface.

103. The path planning method according to claim 98 or 102, characterized in that: The relative position relationship between the target surface and the movable platform is determined by information obtained by a distance measuring sensor.

104. The path planning method according to claim 98, characterized in that: The automatically generating a reference surface based on the target surface includes: Determine the geographic location information of the target surface based on the pre-generated three-mode model; and The reference surface is automatically generated based on the geographic location information of the target surface.

105. The path planning method according to claim 98, characterized in that: Also includes: Adjusting the reference surface based on a positional relationship between the reference surface and the target object in the image so that the adjusted reference surface coincides with the target surface or a deviation between the adjusted reference surface and the target surface is less than a preset value; and A target operating path of the movable platform is generated based on the adjusted reference surface.

106. The path planning method according to claim 98, characterized in that: Also includes: In response to a user's adjustment operation on the reference surface, adjusting the reference surface so that the adjusted reference surface coincides with the target surface or a deviation between the adjusted reference surface and the target surface is less than a preset value; as well as Based on the adjusted reference surface, an operation path of the movable platform is generated.

107. The path planning method according to claim 105 or 106, characterized in that: The adjusting the reference surface includes: The geometric information of the reference surface is adjusted.

108. The path planning method according to claim 107, characterized in that: The geometric information of the reference surface includes at least one of the following: the area of ​​the reference surface, the shape of the reference surface, and the inclination angle of the reference surface.

109. The path planning method according to claim 98, characterized in that: At least a portion of the working path is generated on the reference surface.

110. The path planning method according to claim 98, characterized in that: At least a portion of the working path is substantially parallel to the reference surface.

111. The path planning method according to claim 98, characterized in that: The method further comprises: Adjusting the operation path based on the positional relationship between the operation path and each object in the image, and displaying the adjusted operation path in the image; or In response to a user's adjustment operation on the work path, the work path is adjusted, and the adjusted work path is displayed in the image.

112. The path planning method according to claim 98, characterized in that: The method further comprises: Based on the positional relationship between the operation path and each object in the image, a path segment with a collision risk in the operation path is determined, and the path segment with a collision risk is marked.

113. The path planning method according to claim 98, characterized in that: The method further comprises: Changing the image, where the changed image has a different viewing angle than the image before the change; In the changed image, the reference surface continues to be displayed.

114. The path planning method according to claim 98, characterized in that: The image is a real image, and the reference surface is a virtual image superimposed on the real image.

115. The path planning method according to claim 98, characterized in that: The image and the reference surface are virtual images.

116. The path planning method according to claim 98, characterized in that: The image is a 2D image or a 3D image acquired based on an image sensor; or, the image is a 2D image or a 3D image generated based on sensing information of a ranging sensor.

117. The path planning method according to claim 116, characterized in that: The image sensor and / or the distance measuring sensor is mounted on the movable platform.

118. A method for controlling a movable platform, characterized in that: include: In response to a user's operation, controlling the movable platform to move to the vicinity of a target object, wherein the movable platform is equipped with an image sensor and a ranging sensor; displaying an image of a target surface including the target object acquired by the image sensor; determining a reference surface in the image based at least on the sensing information of the ranging sensor, wherein position information of the reference surface is associated with position information of the target surface; In response to a user's adjustment operation on the reference surface, adjusting the reference surface in the image to obtain an adjusted reference surface; generating an operating path of the movable platform based on the adjusted reference surface; and The movable platform is controlled to move along the working path.

119. The control method according to claim 118, characterized in that: The step of adjusting the reference surface in the image in response to a user's adjustment operation on the reference surface comprises: In response to a user's adjustment operation on the reference surface, geometric information of the reference surface is adjusted in the image.

120. The control method according to claim 119, characterized in that: The geometric information of the reference surface includes at least one of the following: the area of ​​the reference surface, the shape of the reference surface, and the inclination angle of the reference surface.

121. The control method according to any one of claims 118 to 120, characterized in that: The reference surface is substantially parallel to at least a portion of the target surface.

122. The control method according to any one of claims 118 to 120, characterized in that: At least part of the working path is generated on the reference surface; or, At least a portion of the working path is substantially parallel to the reference surface.

123. The control method according to any one of claims 118 to 120, characterized in that: The method further comprises: Adjusting the operation path based on the positional relationship between the operation path and each object in the image, and displaying the adjusted operation path in the image; or In response to a user's adjustment operation on the work path, the work path is adjusted, and the adjusted work path is displayed in the image.

124. The control method according to any one of claims 118 to 120, characterized in that: The method further comprises: Based on the positional relationship between the operation path and each object in the image, a path segment with a collision risk in the operation path is determined, and the path segment with a collision risk is marked.

125. The control method according to any one of claims 118 to 120, characterized in that: The method further comprises: Changing the image, where the changed image has a different viewing angle than the image before the change; In the changed image, the reference surface continues to be displayed.

126. The control method according to any one of claims 118 to 120, characterized in that: The determining of a reference surface in the image based at least on the sensing information of the ranging sensor includes: determining the distance between the target surface and the movable platform based on information acquired by the ranging sensor; and The geographic location information of the reference surface is determined based on the distance between the target surface and the movable platform.

127. The control method according to claim 126, characterized in that: The determining of a reference surface in the image based at least on the sensing information of the ranging sensor further comprises: The geographic location information of the reference surface is determined based on a ranging direction for determining a distance between the target surface and the movable platform.

128. The control method according to claim 126, characterized in that: The geographical location information of the reference surface is determined by the information acquired by the ranging sensor and the geographical location information of the movable platform.

129. The control method according to any one of claims 118 to 120, characterized in that: The image is a real image, and the reference surface is a virtual image superimposed on the real image.

130. The control method according to any one of claims 118 to 120, characterized in that: The image and the reference surface are virtual images.

131. A path planning device, characterized in that: The path planning device includes a memory and a processor, the memory is used to store a computer program, and the processor is used to execute the computer program and implement the path planning method according to any one of claims 1 to 117 when executing the computer program.

132. A control device, characterized in that: The control device includes a memory and a processor, the memory is used to store a computer program, and the processor is used to execute the computer program and implement the control method according to any one of claims 118 to 130 when executing the computer program.

133. A control terminal, characterized in that: The control terminal includes a memory and a processor, the memory is used to store a computer program, and the processor is used to execute the computer program and, when executing the computer program, implement the path planning method as described in any one of claims 1 to 117 or the control method as described in any one of claims 118 to 130.

134. A storage medium for computer-readable storage, characterized in that: The storage medium stores a computer program, which, when executed by a processor, enables the processor to implement the path planning method according to any one of claims 1 to 117 or the control method according to any one of claims 118 to 130.

Images