EVTOL obstacle avoidance method and device, equipment and medium

Abstract

The invention relates to the technical field of flight navigation, in particular to an eVTOL obstacle avoidance method and device, equipment and a medium. The method comprises the following steps: acquiring a surrounding environment image through flight equipment, inputting a preset obstacle recognition model to obtain an obstacle type, if a target type is included, extracting corresponding image depth information, generating local three-dimensional point clouds by combining internal and external parameter back projection of acquisition equipment, summarizing all the local three-dimensional point clouds to update a historical three-dimensional map at the previous moment, and obtaining the target type. And planning an obstacle avoidance path from the current position to the target position according to the updated map, and executing to complete obstacle avoidance. Therefore, the obstacle and the position of the obstacle can be determined in real time in a complex flight environment so as to accurately complete obstacle avoidance.

Description

obstacle avoidance methods, devices, equipment and media for eVTOL Technical Field Technical Field

[0001] This application relates to the field of flight navigation technology, and in particular to an obstacle avoidance method, device, equipment and medium for eVTOL. Background Technology Background Technology

[0002] In recent years, electric vertical takeoff and landing (eVTOL) aircraft have become a research hotspot in the aviation field due to their enormous application potential in urban air traffic, logistics delivery, and emergency rescue. eVTOLs can take off and land vertically in confined spaces without the need for long runways, giving them exceptional flexibility in urban environments. However, obstacle avoidance during flight remains a key challenge for ensuring safe flight for eVTOLs. While various related technologies have been applied, they all have certain limitations.

[0003] Currently, most obstacle avoidance methods acquire environmental images through cameras and then use image processing and machine learning algorithms to identify obstacles. For example, cascaded classifiers based on Haar features can identify specific types of obstacles, such as buildings and utility poles. However, this approach is highly sensitive to lighting conditions; image quality is severely affected in direct sunlight or low-light environments, leading to a decrease in recognition accuracy. Furthermore, traditional visual recognition algorithms can only determine the approximate location and category of obstacles, but cannot accurately calculate the distance to obstacles, which is far from sufficient for obstacle avoidance decisions in eVTOL.

[0004] Therefore, how to determine the position of obstacles and oneself in real time in complex flight environments in order to accurately complete obstacle avoidance has become an urgent problem to be solved. Summary of the Invention Summary of the Invention

[0005] In view of this, embodiments of this application provide an obstacle avoidance method, apparatus, device and medium for eVTOL, to solve the problem of how to determine the position of obstacles and oneself in real time in complex flight environments in order to accurately complete obstacle avoidance.

[0006] In a first aspect, embodiments of this application provide an obstacle avoidance method for eVTOL, comprising: acquiring at least one environmental image around the flight equipment at the current moment; inputting each environmental image into a preset obstacle recognition model and outputting an obstacle type representing each obstacle in the corresponding environmental image; if the obstacle type includes a target type, then for any environmental image, extracting the depth information of the environmental image; performing back projection on the environmental image based on the depth information and the intrinsic and extrinsic parameters of the acquisition device corresponding to the environmental image to obtain a local three-dimensional point cloud corresponding to the environmental image; traversing all environmental images to obtain all local three-dimensional point clouds; updating the historical three-dimensional map of the previous moment based on all local three-dimensional point clouds to obtain an updated three-dimensional map; planning an obstacle avoidance path from the current position to the target position based on the updated three-dimensional map to obtain an obstacle avoidance path; and executing the obstacle avoidance path to complete obstacle avoidance for the flight equipment.

[0007] Secondly, an embodiment of this application provides an obstacle avoidance device for eVTOL, comprising: an obstacle type output module, configured to acquire at least one environmental image around the flight equipment at the current moment, input each environmental image into a preset obstacle recognition model, and output an obstacle type representing each obstacle in the corresponding environmental image; a local point cloud generation module, configured to, if the obstacle type includes a target type, extract the depth information of any environmental image, and back-project the environmental image according to the depth information and the intrinsic and extrinsic parameters of the acquisition device corresponding to the environmental image to obtain a local three-dimensional point cloud corresponding to the environmental image; a map update module, configured to traverse all environmental images to obtain all local three-dimensional point clouds, and update the historical three-dimensional map of the previous moment at the current moment according to all local three-dimensional point clouds to obtain an updated three-dimensional map; and an obstacle avoidance module, configured to, according to the updated three-dimensional map, plan an obstacle avoidance path from the current position to the target position at the current moment to obtain an obstacle avoidance path, and execute the obstacle avoidance path to complete the obstacle avoidance of the flight equipment.

[0008] Thirdly, embodiments of this application provide a computer device, the computer device including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the obstacle avoidance method of eVTOL as described in the first aspect.

[0009] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the obstacle avoidance method for eVTOL as described in the first aspect.

[0010] The beneficial effects of this application embodiment compared with the prior art are as follows: In this application, at least one environmental image around the flight equipment at the current moment is acquired, each environmental image is input into a preset obstacle recognition model, and the obstacle type representing each obstacle in the corresponding environmental image is output. If the obstacle type includes the target type, the depth information of the environmental image is extracted for any environmental image. Based on the depth information and the internal and external parameters of the acquisition device corresponding to the environmental image, the environmental image is back-projected to obtain the local three-dimensional point cloud of the corresponding environmental image. All environmental images are traversed to obtain all local three-dimensional point clouds. Based on all local three-dimensional point clouds, the historical three-dimensional map of the previous moment is updated to obtain the updated three-dimensional map. Based on the updated three-dimensional map, the obstacle avoidance path from the current position to the target position is planned to obtain the obstacle avoidance path. The obstacle avoidance path is executed to complete the obstacle avoidance of the flight equipment. The system acquires images of the surrounding environment using flight equipment, inputs them into a preset obstacle recognition model to determine obstacle types, and if a target type is included, extracts the corresponding image depth information and combines it with the intrinsic and extrinsic parameters of the acquisition equipment for back-projection to generate a local 3D point cloud. All local 3D point clouds are then aggregated to update the historical 3D map from the previous moment. Based on the updated map, an obstacle avoidance path from the current position to the target position is planned and executed to complete obstacle avoidance. This allows for real-time determination of obstacle and flight position in complex flight environments, enabling accurate obstacle avoidance. Attached Figure Description Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 is a schematic diagram of an application environment for an obstacle avoidance method for eVTOL provided in Embodiment 1 of this application; Figure 2 is a schematic flowchart of an obstacle avoidance method for eVTOL provided in Embodiment 2 of this application; Figure 3 is a schematic flowchart of an obstacle avoidance method for eVTOL provided in Embodiment 3 of this application; Figure 4 is a schematic flowchart of an obstacle avoidance method for eVTOL provided in Embodiment 4 of this application; Figure 5 is a schematic flowchart of an obstacle avoidance method for eVTOL provided in Embodiment 5 of this application; Figure 6 is a schematic structural diagram of an obstacle avoidance device for eVTOL provided in Embodiment 6 of this application; Figure 7 is a schematic structural diagram of a computer device provided in Embodiment 7 of this application. Detailed Implementation Detailed Implementation

[0013] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0014] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0015] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0016] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0017] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0018] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0019] It should be understood that the sequence number of each step in the following embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0020] To illustrate the technical solution of this application, specific embodiments are described below.

[0021] The first embodiment of this application provides an obstacle avoidance method for eVTOL, which can be applied in the application environment shown in Figure 1. In this method, the client and the server are connected and communicate. The user can provide the conditions, requirements and operation instructions for obstacle avoidance of eVTOL through the operation of the client. The server is used to control the obstacle avoidance method of eVTOL according to the control instructions sent by the client.

[0022] The client side includes, but is not limited to, PDAs, desktop computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, cloud terminal devices, and personal digital assistants (PDAs). The server side can be a standalone server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0023] Referring to Figure 2, it is a flowchart illustrating an obstacle avoidance method for eVTOL provided in Embodiment 2 of this application. The above-mentioned obstacle avoidance method for eVTOL can be applied to the server in Figure 1.

[0024] As shown in Figure 2, the obstacle avoidance method of this eVTOL may include the following steps: Step S201, acquire at least one environmental image around the flight equipment at the current moment, input each environmental image into a preset obstacle recognition model, and output the obstacle type representing each obstacle in the corresponding environmental image.

[0025] Optionally, after acquiring at least one environmental image of the flight equipment at the current moment, the following steps may be included: correcting the environmental image according to preset camera intrinsic parameters to obtain a corrected image; optimizing the corrected image to obtain an optimized image; inputting each optimized image into a preset obstacle recognition model to output an obstacle type representing each obstacle in the corresponding environmental image.

[0026] Optionally, the step of inputting each environmental image into a preset obstacle recognition model and outputting an obstacle type representing each obstacle in the corresponding environmental image may include the following steps: extracting pixels from any of the environmental images to obtain image pixels; classifying and recognizing the image pixels to obtain an obstacle type representing each obstacle in the corresponding environmental image.

[0027] At any given moment, the flight equipment will use its onboard image acquisition devices (such as cameras) to capture at least one image of its surrounding environment, i.e., an environmental image. These environmental images can reflect the current state of the environment in which the flight equipment is located, and may include various obstacles such as buildings, trees, and other aircraft.

[0028] Each acquired environmental image is input into a pre-defined obstacle recognition model. This model is trained on a large amount of image data labeled with obstacles and is capable of recognizing different types of obstacles. The model analyzes and processes the input images, using learned features and patterns to determine the type of each obstacle in the image.

[0029] After processing by the model, the final output represents the obstacle type for each obstacle in the corresponding environmental image. For example, obstacles can be divided into static obstacles (such as tall buildings and utility poles) and dynamic obstacles (such as birds and other flying drones). The acquired environmental image is then corrected based on preset camera intrinsic parameters. Camera intrinsic parameters are parameters describing the internal characteristics of the camera, such as focal length and principal point position. Due to potential errors during camera manufacturing and use, image distortion and other problems may occur. By using camera intrinsic parameters for correction, these distortions can be eliminated, resulting in a more accurate corrected image.

[0030] The corrected images are then optimized, including image enhancement and noise reduction. Image enhancement improves contrast and clarity, making obstacle features more prominent; noise reduction removes noise interference, further improving image quality. The resulting optimized images are then input into a pre-defined obstacle recognition model for obstacle type identification.

[0031] Besides directly inputting images into a model for recognition, pixel extraction and classification can also be used to determine obstacle types. Pixel extraction is performed on any environmental image to obtain image pixels. An image is composed of individual pixels, each with its specific color and brightness value. By extracting image pixels, basic image information can be obtained. The extracted image pixels are then classified. This step divides pixels into different categories based on their color, texture, brightness, and other features, with each category corresponding to an obstacle type. For example, pixels with colors and textures matching tree characteristics are classified as tree obstacles, and pixels matching building characteristics are classified as building obstacles, and so on. In this way, the obstacle type representing each obstacle in the corresponding environmental image is finally obtained.

[0032] Step S202: If the obstacle type includes the target type, then for any environmental image, extract the depth information of the environmental image, and perform back projection on the environmental image according to the depth information and the internal and external parameters of the acquisition device corresponding to the environmental image to obtain the local three-dimensional point cloud corresponding to the environmental image.

[0033] Optionally, after back-projecting the environmental image based on the depth information and the intrinsic and extrinsic parameters of the acquisition device corresponding to the environmental image to obtain a local three-dimensional point cloud corresponding to the environmental image, the following step may be included: removing impurities from the local three-dimensional point cloud according to a preset filtering scheme to obtain an optimized local three-dimensional point cloud.

[0034] When the identified obstacle types include target types (such as specific types of obstacles that may have a significant impact on flight and require special attention, such as tall buildings or dense power lines), in order to more accurately understand the location and distribution of obstacles in three-dimensional space, it is necessary to convert the two-dimensional environmental image into three-dimensional local three-dimensional point cloud data.

[0035] Depth information reflects the distance of each pixel in an image to the camera. Flying devices can acquire depth information in various ways. Common methods include using depth cameras (such as structured light cameras and Time-of-Flight (TOF) cameras), which can directly measure the depth of objects in a scene; or using binocular vision technology, which calculates depth information by utilizing the parallax between images taken by two cameras. Once depth information is acquired, the approximate distances of various objects in the environment image to three-dimensional space can be determined. Camera intrinsic parameters describe the camera's internal optical characteristics, including focal length and principal point position. Intrinsic parameters determine how the camera projects points in three-dimensional space onto a two-dimensional image plane; conversely, during back-projection, intrinsic parameters help us recover three-dimensional space coordinates from two-dimensional image coordinates. Camera extrinsic parameters represent the camera's position and attitude in three-dimensional space, including translation and rotation information. Using extrinsic parameters, we can transform three-dimensional points based on the camera coordinate system to the world coordinate system, thereby determining the position of obstacles in actual space.

[0036] Backprojection is the process of mapping each pixel in a 2D image back to 3D space, combining its depth information with the camera's intrinsic and extrinsic parameters. Specifically, for each pixel in the image, the coordinates of its corresponding point in 3D space are calculated based on its depth value and the camera's intrinsic and extrinsic parameters. Combining the 3D points corresponding to all pixels in the image yields a local 3D point cloud of the corresponding environment. A 3D point cloud is a collection of numerous 3D coordinate points that can visually represent the 3D shape and position of objects in the environment.

[0037] After obtaining a local 3D point cloud, due to factors such as measurement errors and noise, the point cloud may contain some impurity points. These impurity points may interfere with subsequent processing and analysis, thus requiring filtering. The preset filtering scheme is a series of filtering algorithms designed based on actual needs and the characteristics of the point cloud data. Gaussian filtering, median filtering, statistical filtering, etc., can be used.

[0038] By executing a preset filtering scheme, impurity points in the point cloud are removed, resulting in an optimized local 3D point cloud. The optimized point cloud data is cleaner and more accurate, more realistically reflecting the 3D structure of objects in the environment, providing a more reliable foundation for subsequent 3D map updates and obstacle avoidance path planning.

[0039] Step S203: Traverse all environmental images to obtain all local 3D point clouds. Based on all local 3D point clouds, update the historical 3D map of the previous moment at the current moment to obtain the updated 3D map.

[0040] The flight equipment may acquire multiple environmental images from different angles and positions at any given moment. Each environmental image has been processed in the preceding steps (such as step S202) to obtain a corresponding local 3D point cloud. Traversing all these environmental images means processing each image sequentially and collecting their respective local 3D point clouds. These local 3D point clouds reflect the 3D environmental information of different areas around the flight equipment.

[0041] Historical 3D maps are constructed by the flight equipment at a previous moment, recording the 3D positional information of obstacles and other features in the flight environment at that time. However, as the flight equipment moves and the environment dynamically changes, historical 3D maps may no longer accurately reflect the current environmental conditions. Therefore, it is necessary to update them using all local 3D point clouds acquired at the current moment.

[0042] The process involves fusing current local 3D point cloud data with historical 3D maps. Since point cloud data from different times may have inconsistent coordinate systems, a coordinate transformation is first performed to unify them into a single coordinate system. Then, the new point cloud data is added to the historical map, ensuring the map contains more up-to-date environmental information.

[0043] During the fusion process, duplicate data may occur between local 3D point clouds and historical maps; that is, point cloud information for certain areas may appear in both the old and new data. In such cases, this duplicate data needs to be processed. For example, by comparing the location and attributes of points, the most up-to-date and accurate data can be retained, and redundant information can be removed to avoid excessive map data expansion. If dynamic obstacles exist in the environment (such as moving vehicles, flying birds, etc.), or new obstacles appear (such as temporary buildings), or some obstacles are removed, the local 3D point cloud will reflect these changes. When updating the historical map, the information in the corresponding areas of the map needs to be modified or supplemented based on the new point cloud data to ensure that the map accurately reflects the actual situation of the current environment.

[0044] After the aforementioned data fusion, repetitive data processing, and dynamic updates, an updated 3D map is obtained. This map integrates the latest information about the surrounding environment of the flight equipment at the current moment, including the location, shape, and type of obstacles. The updated 3D map provides a more accurate basis for subsequent obstacle avoidance path planning. The flight equipment can use this map to plan an obstacle avoidance path from its current position to the target position, thereby completing the flight mission more safely and efficiently.

[0045] Step S204: Based on the updated 3D map, plan the obstacle avoidance path from the current position to the target position at the current moment to obtain the obstacle avoidance path, and execute the obstacle avoidance path to complete the obstacle avoidance of the flight equipment.

[0046] The updated 3D map contains the latest information on the environment surrounding the flight equipment, accurately showing the location, shape, size, and distribution of obstacles. This information is crucial for path planning, as the planned path must avoid obstacles marked on the map to prevent collisions during flight.

[0047] The starting point is the current location of the flight equipment, and the ending point is the pre-set target location. Both locations have specific coordinates in the coordinate system used to update the 3D map.

[0048] Path planning algorithms can include A* algorithm, Dijkstra's algorithm, RRT (Rapid Exploratory Random Tree) algorithm, etc.

[0049] The A* algorithm uses a heuristic function to estimate the cost from the current node to the target node, thereby guiding the search in the most promising direction and quickly finding a better obstacle avoidance path.

[0050] Dijkstra's algorithm traverses all possible paths on the map, calculates the shortest path from the starting point to each node, and finally finds the shortest obstacle avoidance path from the starting point to the target point. However, its computational cost is relatively high, especially when the map is large.

[0051] The RRT algorithm is a sampling-based path planning algorithm that explores feasible paths by randomly sampling nodes in a map and progressively expanding the tree structure. The RRT algorithm can quickly find a feasible path in complex environments, but the path found may not be optimal.

[0052] During path planning, the constraints of the flight equipment itself must also be considered, such as maximum flight speed, minimum turning radius, and flight altitude limits. The planned path must meet these constraints to ensure that the flight equipment can actually execute the path.

[0053] After calculations by the path planning algorithm and consideration of the constraints of the flight equipment, an obstacle avoidance path from the current position to the target position is finally obtained. This path usually consists of a series of three-dimensional coordinate points, and the flight equipment can fly according to these points in sequence to avoid obstacles.

[0054] The flight control system of the flight equipment controls its attitude, speed, and direction according to the planned obstacle avoidance path, ensuring it flies along the path. During flight, the equipment continuously senses its surrounding environment, updates the 3D map in real time, and adjusts the path based on the new map information to cope with possible new obstacles or environmental changes, ensuring safe obstacle avoidance flight and ultimately reaching the target location.

[0055] This application acquires at least one environmental image of the surroundings of the flight equipment at the current moment. Each environmental image is input into a preset obstacle recognition model, and the model outputs the obstacle type representing each obstacle in the corresponding environmental image. If the obstacle type includes a target type, the depth information of the environmental image is extracted for any given environmental image. Based on the depth information and the intrinsic and extrinsic parameters of the acquisition device corresponding to the environmental image, the environmental image is back-projected to obtain a local 3D point cloud. All environmental images are traversed to obtain all local 3D point clouds. Based on all local 3D point clouds, the historical 3D map of the previous moment is updated to obtain an updated 3D map. Based on the updated 3D map, an obstacle avoidance path from the current position to the target position is planned to obtain an obstacle avoidance path. The obstacle avoidance path is then executed to complete obstacle avoidance for the flight equipment. The flight equipment acquires surrounding environmental images, inputs them into a preset obstacle recognition model to obtain obstacle types, and if a target type is included, extracts the corresponding image depth information and back-projects it using the intrinsic and extrinsic parameters of the acquisition device to generate a local 3D point cloud. All local 3D point clouds are summarized to update the historical 3D map of the previous moment. An obstacle avoidance path is planned and executed based on the updated map to complete obstacle avoidance. This allows for real-time determination of obstacle and self-position in complex flight environments, enabling accurate obstacle avoidance.

[0056] Referring to Figure 3, it is a flowchart illustrating an obstacle avoidance method for eVTOL provided in Embodiment 3 of this application. As shown in Figure 3, after the obstacle type of each obstacle in the output characterizing the corresponding environmental image in step S201, the following steps may be included: Step S301, analyzing the obstacles in the obstacle type, and if the obstacle has predictable motion, determining that the obstacle type includes a target class; Step S302, for any environmental image, extracting the depth information of the environmental image, and performing back projection on the environmental image based on the depth information and the intrinsic and extrinsic parameters of the corresponding acquisition device to obtain a local three-dimensional point cloud corresponding to the environmental image.

[0057] Once the obstacle type for each obstacle in the environmental image is obtained, these obstacles need to be analyzed one by one. The analysis here mainly focuses on the motion characteristics of the obstacles.

[0058] Predictable motion refers to obstacles whose motion patterns can be predicted using some method. For example, the trajectories and speeds of satellites orbiting on fixed paths and vehicles traveling on regular schedules can be modeled and predicted under certain conditions. However, obstacles like plastic bags drifting in the wind and birds flying irregularly cannot be predicted using simple models and therefore do not fall under the category of obstacles with predictable motion.

[0059] Once an obstacle is determined to have predictable motion, it is considered to fall under the target category. This type of obstacle is defined as a target because predictable obstacles tend to be more regular; focusing on tracking and handling them allows for more effective obstacle avoidance path planning for flight equipment, improving flight safety and efficiency.

[0060] Depth information reflects the distance between the object corresponding to each pixel in an environmental image and the image acquisition device (such as a camera). Depth information can be obtained in various ways, such as using dedicated depth cameras (such as structured light cameras or time-of-flight (TOF) cameras), which can directly measure the depth of pixels, or using binocular vision technology, which uses the parallax of images taken by two cameras to calculate the depth of pixels.

[0061] Camera intrinsic parameters describe the camera's internal optical characteristics, such as focal length and principal point position. Intrinsic parameters determine how the camera projects points in three-dimensional space onto a two-dimensional image plane; during backprojection, they are used to restore the corresponding positions of pixels in the two-dimensional image to their positions in three-dimensional space.

[0062] Camera extrinsic parameters represent the camera's position and orientation in three-dimensional space, including translation and rotation information. Using extrinsic parameters, three-dimensional points based on the camera coordinate system can be transformed to a unified world coordinate system, thereby determining the accurate location of obstacles in actual space.

[0063] Back projection is a process of converting two-dimensional image information into three-dimensional spatial information. By combining the depth information of each pixel with the camera's intrinsic and extrinsic parameters, the coordinates of each pixel in the image in three-dimensional space can be calculated. Combining the three-dimensional coordinates of all pixels forms a local three-dimensional point cloud of the corresponding environment image. This three-dimensional point cloud can intuitively display the three-dimensional shape and position of obstacles in the environment, providing basic data for further analysis and processing of target-type obstacles.

[0064] Based on the identification of obstacle types, this application focuses on obstacles with predictable movement and obtains more accurate three-dimensional information by constructing local three-dimensional point clouds, providing a more reliable basis for obstacle avoidance operations of flight equipment.

[0065] Referring to Figure 4, it is a flowchart illustrating an obstacle avoidance method for eVTOL provided in Embodiment 4 of this application. As shown in Figure 4, after the output in step S201 characterizes the obstacle type of each obstacle in the corresponding environmental image, the following steps may be included: Step S401, analyzing the obstacles in the obstacle type; Step S402, if the obstacle has unpredictable movement, generating a waiting command for the flight device to perform obstacle avoidance operation.

[0066] In step S201, the environmental image has been processed using image processing and target detection technologies, outputting the type information of each obstacle, such as buildings, pedestrians, vehicles, and birds. This step focuses on analyzing the motion state and patterns of the obstacles. The motion of obstacles is divided into two categories: predictable motion and unpredictable motion. Obstacles with predictable motion exhibit certain regularities in their movement, which can be predicted using mathematical models, physical laws, or historical data. Examples include trains traveling on fixed tracks and airplanes flying along predetermined routes. Obstacles with unpredictable motion, on the other hand, have trajectories, speeds, and directions that are difficult to predict accurately in advance and may be affected by various random factors. Examples include birds and bees flying randomly in the sky.

[0067] Multiple image sequences can be compared to track the obstacle's position changes at different times, calculating its velocity, acceleration, and other motion parameters to determine if its motion is predictable. Alternatively, sensor data, such as radar and lidar, can be combined to obtain more precise motion information about the obstacle.

[0068] Based on the above judgment results, the flight equipment's control system generates a waiting command. This command is sent to the flight equipment's actuators, requesting the flight equipment to suspend its current flight mission. Upon receiving the waiting command, the flight equipment immediately stops its planned flight path, maintaining its current position or hovering in the air. During the waiting process, the flight equipment continuously monitors the surrounding environment, acquiring the latest information on obstacles in real time. When an unpredictable obstacle leaves the flight path, or when its movement becomes predictable, the flight equipment replans its path and continues flying towards the target location, thereby avoiding collisions with obstacles and achieving effective obstacle avoidance.

[0069] Referring to Figure 5, it is a flowchart illustrating an obstacle avoidance method for eVTOL provided in Embodiment 5 of this application. As shown in Figure 5, step S204, which involves planning an obstacle avoidance path from the current position to the target position based on the updated 3D map, obtaining the obstacle avoidance path, and executing the obstacle avoidance path to complete obstacle avoidance for the flight equipment, may include the following steps: Step S501, determining the historical position of the flight equipment at the previous moment based on the historical 3D map of the previous moment; Step S502, determining the position change of the flight equipment based on the historical position and the current position, and determining the current pose based on the position change; Step S503, obtaining the obstacle avoidance path based on the target position and the current pose, and executing the obstacle avoidance path to complete obstacle avoidance for the flight equipment.

[0070] The historical 3D map records the 3D information of the environment in which the flight equipment was located at the previous moment, including the distribution of surrounding obstacles and the possible location of the flight equipment itself. This map is like a "time slice," preserving the scene at a certain moment in the past.

[0071] By analyzing historical 3D maps and combining them with the location markers or relevant data records of the flight equipment on the map, the specific location of the flight equipment at the previous moment can be determined. This location is an important reference point for subsequent calculations of position changes. By comparing the current position with the historical position and using calculation methods such as coordinate difference, the position change of the flight equipment between these two moments can be obtained, including displacement in the X, Y, and Z axes in 3D space.

[0072] Pose includes not only position but also the attitude of the flight equipment (such as pitch, yaw, and roll angles). Position change information helps infer the flight equipment's direction and trend of motion. Combined with data from the flight equipment's own attitude sensors (such as gyroscopes), the current pose of the flight equipment is comprehensively determined. Pose information is crucial for accurate obstacle avoidance path planning, as different poses affect the flight equipment's movement and operational space. Based on the known target position and the flight equipment's current pose, and combined with updated obstacle distribution information in the 3D map, a suitable path planning algorithm (such as A* algorithm, Dijkstra's algorithm, etc.) is used to find a path from the current position to the target position that avoids obstacles. During the planning process, the impact of the flight equipment's pose on the path needs to be considered, such as the turning radius and flight speed constraints under certain poses.

[0073] The planned obstacle avoidance path is sent to the flight equipment's control system. The control system controls the flight equipment's motors, servos, and other actuators based on the path information, adjusting the flight equipment's attitude and speed so that the flight equipment flies along the planned path, thereby completing the obstacle avoidance task and finally reaching the target location.

[0074] In this embodiment, by fully utilizing historical and current information, the position and attitude of the flight equipment are gradually determined, thereby planning a reasonable obstacle avoidance path. This path planning method based on multi-faceted information can better adapt to complex and ever-changing flight environments, improving the accuracy and reliability of obstacle avoidance for the flight equipment.

[0075] Corresponding to the obstacle avoidance method of eVTOL in the above embodiments, Figure 6 shows a structural block diagram of the obstacle avoidance device of eVTOL provided in Embodiment Six of this application. The above-mentioned obstacle avoidance device of eVTOL can be applied to the server in Figure 1. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0076] Referring to Figure 6, the obstacle avoidance device of this eVTOL includes: an obstacle type output module 61, used to acquire at least one environmental image around the flight equipment at the current moment, input each environmental image into a preset obstacle recognition model, and output an obstacle type representing each obstacle in the corresponding environmental image; a local point cloud generation module 62, used to extract the depth information of any environmental image if the obstacle type includes a target type, and back-project the environmental image according to the depth information and the intrinsic and extrinsic parameters of the acquisition device corresponding to the environmental image to obtain a local three-dimensional point cloud corresponding to the environmental image; a map update module 63, used to traverse all environmental images to obtain all local three-dimensional point clouds, and update the historical three-dimensional map of the previous moment based on all local three-dimensional point clouds to obtain an updated three-dimensional map; and an obstacle avoidance module 64, used to plan an obstacle avoidance path from the current position to the target position based on the updated three-dimensional map, obtain an obstacle avoidance path, and execute the obstacle avoidance path to complete the obstacle avoidance of the flight equipment.

[0077] Optionally, the obstacle avoidance device of the eVTOL includes: a correction module for correcting the environmental image according to preset camera intrinsic parameters to obtain a corrected image; and a recognition module for optimizing the corrected image to obtain an optimized image, inputting each optimized image into a preset obstacle recognition model, and outputting an obstacle type representing each obstacle in the corresponding environmental image.

[0078] Optionally, the obstacle type output module 61 includes: a pixel extraction unit for extracting pixels from any of the environmental images to obtain image pixels; and a classification unit for classifying and identifying the image pixels to obtain the obstacle type representing each obstacle in the corresponding environmental image.

[0079] Optionally, the obstacle avoidance device of the eVTOL includes: an obstacle analysis module, used to analyze the obstacles in the obstacle type after the output characterizes the obstacle type of each obstacle in the corresponding environmental image, and if the obstacle has predictable motion, determine that the obstacle type includes a target class; and an information analysis module, used to extract the depth information of any environmental image, and perform back projection on the environmental image according to the depth information and the intrinsic and extrinsic parameters of the corresponding acquisition device to obtain a local three-dimensional point cloud of the corresponding environmental image.

[0080] Optionally, the obstacle avoidance device of the eVTOL further includes: a type analysis module, used to analyze the obstacles in the obstacle type after the output characterizing the obstacle type of each obstacle in the corresponding environmental image; and a waiting instruction generation module, used to generate a waiting instruction for the flight device to perform obstacle avoidance operation if the obstacle has unpredictable motion.

[0081] Optionally, the obstacle avoidance module 64 includes: a position determination unit, used to determine the historical position of the flight device at the previous moment based on the historical 3D map of the previous moment; a pose determination unit, used to determine the position change of the flight device based on the historical position and the current position, and to determine the current pose based on the position change; and an execution unit, used to obtain an obstacle avoidance path based on the target position and the current pose, and to execute the obstacle avoidance path to complete the obstacle avoidance of the flight device.

[0082] Optionally, the obstacle avoidance device of the eVTOL further includes: an optimization module, used to perform impurity removal on the local three-dimensional point cloud according to a preset filtering scheme after back-projecting the environmental image based on the depth information and the intrinsic and extrinsic parameters of the acquisition device corresponding to the environmental image to obtain a local three-dimensional point cloud corresponding to the environmental image, thereby obtaining an optimized local three-dimensional point cloud.

[0083] It should be noted that the information interaction and execution process between the above modules, units, and sub-units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0084] Figure 7 is a schematic diagram of the structure of a computer device provided in Embodiment 7 of this application. As shown in Figure 7, the computer device of this embodiment includes: at least one processor (only one is shown in Figure 7), a memory, and a computer program stored in the memory and executable on at least one processor. When the processor executes the computer program, it implements the steps in any of the above-described eVTOL obstacle avoidance method embodiments.

[0085] The computer device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that Figure 7 is merely an example of a computer device and does not constitute a limitation thereof. A computer device may include more or fewer components than shown, or a combination of certain components, or different components, such as a network interface, a display screen, and input devices.

[0086] The processor referred to can be a CPU, but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0087] Memory includes readable storage media, internal memory, etc., wherein internal memory can be the RAM of a computer device, providing an environment for the operation of the operating system and computer-readable instructions stored in the readable storage media. The readable storage media can be the hard drive of a computer device, or in other embodiments, it can be an external storage device of the computer device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, memory can include both internal storage units and external storage devices of the computer device. Memory is used to store the operating system, applications, bootloader, data, and other programs, such as program code for computer programs. Memory can also be used to temporarily store data that has been output or will be output.

[0088] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium can include at least: any entity or device capable of carrying computer program code, a recording medium, a computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0089] The implementation of all or part of the processes in the methods of the above embodiments can also be accomplished by a computer program product. When the computer program product is run on a computer device, it enables the computer device to execute the steps in the above method embodiments.

[0090] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0091] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0092] In the embodiments provided in this application, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0093] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0094] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An obstacle avoidance method for eVTOL, characterized in that, include: Acquire at least one environmental image of the surroundings of the flight equipment at the current moment, input each environmental image into a preset obstacle recognition model, and output the obstacle type representing each obstacle in the corresponding environmental image; If the obstacle type includes a target type, then for any environmental image, the depth information of the environmental image is extracted. Based on the depth information and the intrinsic and extrinsic parameters of the acquisition device corresponding to the environmental image, the environmental image is back-projected to obtain a local 3D point cloud corresponding to the environmental image. All environmental images are traversed to obtain all local 3D point clouds. Based on all local 3D point clouds, the historical 3D map of the previous moment is updated to obtain an updated 3D map. Based on the updated 3D map, an obstacle avoidance path from the current position to the target position is planned to obtain an obstacle avoidance path. The obstacle avoidance path is then executed to complete the obstacle avoidance of the flight equipment.

2. The obstacle avoidance method for eVTOL according to claim 1, characterized in that, After acquiring at least one environmental image of the flight equipment at the current moment, the method further includes: correcting the environmental image according to preset camera intrinsic parameters to obtain a corrected image; optimizing the corrected image to obtain an optimized image; inputting each optimized image into a preset obstacle recognition model and outputting an obstacle type representing each obstacle in the corresponding environmental image.

3. The obstacle avoidance method for eVTOL according to claim 1, characterized in that, The step of inputting each environmental image into a preset obstacle recognition model and outputting an obstacle type representing each obstacle in the corresponding environmental image includes: extracting pixels from any of the environmental images to obtain image pixels; classifying and recognizing the image pixels to obtain an obstacle type representing each obstacle in the corresponding environmental image.

4. The obstacle avoidance method for eVTOL according to claim 1, characterized in that, After the output characterizes the obstacle type of each obstacle in the corresponding environmental image, the method further includes: analyzing the obstacles in the obstacle type; if the obstacle has predictable motion, determining that the obstacle type contains a target class; for any environmental image, extracting the depth information of the environmental image; and performing back projection on the environmental image based on the depth information and the intrinsic and extrinsic parameters of the corresponding acquisition device to obtain a local three-dimensional point cloud of the corresponding environmental image.

5. The obstacle avoidance method for eVTOL according to claim 1, characterized in that, After the output characterizes the obstacle type of each obstacle in the corresponding environmental image, the method further includes: analyzing the obstacles in the obstacle type; if the obstacle is moving unpredictably, generating a waiting command for the flight device to perform obstacle avoidance operation.

6. The obstacle avoidance method for eVTOL according to claim 1, characterized in that, The step of planning an obstacle avoidance path from the current position to the target position based on the updated 3D map, obtaining an obstacle avoidance path, and executing the obstacle avoidance path to complete obstacle avoidance for the flight equipment includes: determining the historical position of the flight equipment at the previous moment based on the historical 3D map at the previous moment; determining the position change of the flight equipment based on the historical position and the current position; determining the current pose based on the position change; obtaining an obstacle avoidance path based on the target position and the current pose; and executing the obstacle avoidance path to complete obstacle avoidance for the flight equipment.

7. The obstacle avoidance method for eVTOL according to claim 1, characterized in that, After back-projecting the environmental image based on the depth information and the intrinsic and extrinsic parameters of the acquisition device corresponding to the environmental image to obtain a local three-dimensional point cloud corresponding to the environmental image, the method further includes: removing impurities from the local three-dimensional point cloud according to a preset filtering scheme to obtain an optimized local three-dimensional point cloud.

8. An obstacle avoidance device for eVTOL, characterized in that, include: The obstacle type output module is used to acquire at least one environmental image around the flight equipment at the current moment, input each environmental image into a preset obstacle recognition model, and output the obstacle type representing each obstacle in the corresponding environmental image; the local point cloud generation module is used to extract the depth information of any environmental image if the obstacle type includes the target type, and back-project the environmental image according to the depth information and the intrinsic and extrinsic parameters of the acquisition device corresponding to the environmental image to obtain the local three-dimensional point cloud corresponding to the environmental image; the map update module is used to traverse all environmental images to obtain all local three-dimensional point clouds, and update the historical three-dimensional map of the previous moment according to all local three-dimensional point clouds to obtain an updated three-dimensional map; the obstacle avoidance module is used to plan the obstacle avoidance path from the current position to the target position according to the updated three-dimensional map, obtain the obstacle avoidance path, and execute the obstacle avoidance path to complete the obstacle avoidance of the flight equipment.

9. A computer device, characterized in that, The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the obstacle avoidance method of eVTOL as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the obstacle avoidance method of eVTOL as described in any one of claims 1 to 7.

Images