Unmanned aerial vehicle obstacle avoidance method, device, equipment, storage medium and program product

Abstract

This application discloses a method, apparatus, device, storage medium, and program product for obstacle avoidance by unmanned aerial vehicles (UAVs), belonging to the field of UAV technology. In this application, environmental images and three-dimensional environmental data collected by the UAV are acquired, and a first distance between the UAV and an obstacle is determined based on the three-dimensional environmental data. In response to the first distance being less than or equal to a first distance threshold, the environmental image is input into an image description model to obtain descriptive text for the environmental image, and the descriptive text is input into a risk assessment model to obtain risk assessment text for the obstacle. In response to the risk assessment text indicating a collision risk to the UAV from the obstacle, the UAV is controlled to avoid the obstacle. This application ensures the smoothness and efficiency of UAV flight missions.

Description

Technical Field

[0001] This application relates to the field of unmanned aerial vehicle (UAV) technology, and in particular to a UAV obstacle avoidance method, device, equipment, storage medium, and program product. Background Technology

[0002] With advancements in automatic control and artificial intelligence technologies, mobile robots such as drones and unmanned vehicles have been widely applied in various fields, including logistics and delivery, agricultural plant protection, and security patrol. In these complex scenarios, enabling robots to accurately perceive their environment, identify obstacles, and make safe avoidance decisions is one of the core challenges in achieving fully autonomous operation.

[0003] Currently, mainstream drone obstacle avoidance solutions primarily rely on various sensors to acquire environmental information. These sensors directly measure the distance between the drone and surrounding objects using visual sensors, millimeter-wave radar, lidar, or ultrasonic sensors. When the detected distance falls below a preset safety threshold, the flight control system triggers a pre-defined avoidance strategy, such as hovering, detouring, or climbing.

[0004] However, the aforementioned existing technologies still have significant limitations. First, methods based on pure distance measurement are too mechanical, relying solely on a single distance dimension to judge risk. They fail to understand the specific attributes and potential threats of obstacles (for example, a tree branch swaying in the wind is usually more dangerous than a stationary billboard), which may lead to misjudgments or omissions, resulting in poor obstacle avoidance accuracy. Summary of the Invention

[0005] This application provides a method, apparatus, device, storage medium, and program product for obstacle avoidance of unmanned aerial vehicles (UAVs), which can solve the problem of poor accuracy in obstacle avoidance. The technical solution is as follows: On the one hand, a method for obstacle avoidance by unmanned aerial vehicles (UAVs) is provided, the method comprising: The system acquires environmental images and 3D environmental data collected by the drone, and determines a first distance between the drone and the obstacle based on the 3D environmental data. In response to the first distance being less than or equal to a first distance threshold, the environmental image is input into an image description model to obtain a description text for the environmental image, and the description text is input into a risk assessment model to obtain a risk assessment text for the obstacle; In response to the risk assessment text indicating that the obstacle poses a collision risk to the drone, the drone is controlled to avoid the obstacle.

[0006] In one possible implementation, the risk assessment model includes a generative artificial intelligence model.

[0007] In another possible implementation, inputting the descriptive text into the risk assessment model to obtain a risk assessment text for the obstacle includes: The descriptive text is inserted into the prompt word template to obtain model prompt words. The prompt word template is used to instruct the generative artificial intelligence model to assess the risk of the obstacle based on the descriptive text in order to generate risk assessment text. The model prompts are input into a generative artificial intelligence model to obtain a risk assessment text for the obstacle.

[0008] In another possible implementation, the step of controlling the drone to avoid the obstacle in response to the risk assessment text indicating a collision risk to the drone includes: The risk level corresponding to the obstacle is determined based on the risk assessment text; When the risk level is greater than or equal to the risk level threshold, it is determined that the obstacle poses a collision risk to the drone; The drone is controlled to avoid the obstacle based on the risk level.

[0009] In another possible implementation, controlling the drone to avoid the obstacle based on the risk level includes: Obtain the current flight status parameters of the UAV; Based on the risk level and the designated avoidance strategy library, a target avoidance strategy is determined, wherein the designated avoidance strategy library includes avoidance strategies corresponding to different risk levels, and the avoidance strategy includes at least one of emergency braking strategy, path replanning strategy and altitude adjustment strategy; Based on the target avoidance strategy and the current flight status parameters, flight control commands are generated, and the UAV is controlled to avoid the obstacle based on the flight control commands.

[0010] In another possible implementation, the environmental 3D data includes point cloud data; determining the first distance between the UAV and the obstacle based on the environmental 3D data includes: The geometric boundaries of the obstacle are determined based on the point cloud data; Determine the minimum Euclidean distance between the UAV and the geometric boundary of the obstacle, and use the minimum Euclidean distance as the first distance.

[0011] On the other hand, a drone obstacle avoidance device is provided, the device comprising: The acquisition module is configured to acquire environmental images and three-dimensional environmental data collected by the UAV, and determine a first distance between the UAV and the obstacle based on the three-dimensional environmental data. An assessment module is configured to, in response to the first distance being less than or equal to a first distance threshold, input an environmental image into an image description model to obtain a description text for the environmental image, and input the description text into a risk assessment model to obtain a risk assessment text for the obstacle; A control module is configured to control the drone to avoid the obstacle in response to the risk assessment text indicating that the obstacle poses a collision risk to the drone.

[0012] In one possible implementation, the risk assessment model includes a generative artificial intelligence model.

[0013] In another possible implementation, the evaluation module is used for: The descriptive text is inserted into the prompt word template to obtain model prompt words. The prompt word template is used to instruct the generative artificial intelligence model to assess the risk of the obstacle based on the descriptive text in order to generate risk assessment text. The model prompts are input into a generative artificial intelligence model to obtain a risk assessment text for the obstacle.

[0014] In another possible implementation, the control module is used for: The risk level corresponding to the obstacle is determined based on the risk assessment text; When the risk level is greater than or equal to the risk level threshold, it is determined that the obstacle poses a collision risk to the drone; The drone is controlled to avoid the obstacle based on the risk level.

[0015] In another possible implementation, the control module is used for: Obtain the current flight status parameters of the UAV; Based on the risk level and the designated avoidance strategy library, a target avoidance strategy is determined, wherein the designated avoidance strategy library includes avoidance strategies corresponding to different risk levels, and the avoidance strategy includes at least one of emergency braking strategy, path replanning strategy and altitude adjustment strategy; Based on the target avoidance strategy and the current flight status parameters, flight control commands are generated, and the UAV is controlled to avoid the obstacle based on the flight control commands.

[0016] In another possible implementation, the environmental 3D data includes point cloud data; the acquisition module is used for: The geometric boundaries of the obstacle are determined based on the point cloud data; Determine the minimum Euclidean distance between the UAV and the geometric boundary of the obstacle, and use the minimum Euclidean distance as the first distance.

[0017] On the other hand, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the program to implement the method described in any of the above.

[0018] On the other hand, a non-transitory computer-readable storage medium is provided, the non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method described in any of the preceding claims.

[0019] On the other hand, a computer program product is provided, including computer program instructions that, when run on a computer, cause the computer to perform the method described in any of the preceding claims.

[0020] The beneficial effects of the technical solution provided in this application are as follows: By performing semantic analysis and reasoning on obstacles through image description models and risk assessment models, the categories, states, and potential threat characteristics of obstacles are analyzed, enabling drones to more accurately assess real risks and effectively avoid misjudgments or omissions. Furthermore, by controlling drone avoidance based on the risk level indicated by the risk assessment text, obstacle avoidance accuracy is improved, ensuring the smoothness and efficiency of drone flight missions. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of an implementation environment provided in an embodiment of this application; Figure 2 This is a flowchart of the drone obstacle avoidance method provided in the embodiments of this application; Figure 3 This is a schematic diagram of the structure of the drone obstacle avoidance device provided in the embodiments of this application; Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0024] This disclosure provides an obstacle avoidance method for unmanned aerial vehicles (UAVs). This method can be implemented by a computer device, which may be a terminal device or a server installed inside the UAV. The terminal device may be an onboard computing unit or a flight control computer (FCC). The server may be an application server, a cloud service server, or a server used to perform certain computing tasks, etc. This disclosure uses the example of a terminal device as the computer device for detailed explanation; other cases are similar and will not be described in detail.

[0025] like Figure 1 As shown, the terminal device may include a processor 110, a memory 120, and a communication component 130, etc.

[0026] The processor 110 can be a central processing unit (CPU), which can be used to execute various operation instructions.

[0027] The memory 120 can be various volatile or non-volatile memory, such as solid-state disk (SSD), dynamic random access memory (DRAM), etc. The memory can be used to store pre-stored data, intermediate data, and result data related to the processing.

[0028] The communication component 130 can be a wired network connector, a wireless fidelity (WiFi) module, a Bluetooth module, a cellular communication module, etc. The communication component can be used to transmit data with other devices.

[0029] In some embodiments, such as Figure 2 As shown, the drone obstacle avoidance method includes: S201. Acquire environmental images and three-dimensional environmental data collected by the UAV, and determine a first distance between the UAV and the obstacle based on the three-dimensional environmental data.

[0030] The environmental 3D data can be acquired using millimeter-wave radar, structured light depth cameras, or binocular stereo vision systems. For example, millimeter-wave radar measures distance and velocity by emitting millimeter waves and analyzing the frequency changes of the reflected waves; the generated data can also be represented as a set of points in 3D space. After acquiring the raw 3D data, preprocessing can be performed, such as filtering and noise reduction, and motion distortion correction, to improve data quality.

[0031] Environmental 3D data is used to accurately describe the geometry and spatial relationships of the environment surrounding the drone. Environmental 3D data can also be point cloud data, which is acquired by a lidar sensor mounted on the drone. LiDAR emits a laser beam and measures the time difference from emission to reception after reflection by an obstacle (time-of-flight method) to calculate the distance between the sensor and various points in the environment, thereby generating a set of discrete points with three-dimensional coordinates (X, Y, Z), i.e., point cloud data. Point cloud data can accurately reflect the surface contours and spatial positions of obstacles.

[0032] The environmental images are image data acquired by visual sensors (such as visible light cameras, infrared cameras, etc.) mounted on a drone. These images can be a single image or multiple consecutive frames acquired over a certain period. The environmental images contain rich texture, color, and semantic information. To ensure consistency between the environmental images and 3D spatial information in subsequent analysis, it is necessary to ensure that the visual sensors acquiring the environmental images and the sensors acquiring the 3D environmental data (such as LiDAR) are calibrated and synchronized in time and space. Specifically, the coordinate transformation relationship between the sensors is obtained through joint calibration, and hardware triggering or software timestamps are used to ensure that the environmental image acquired at a certain moment is temporally aligned with the corresponding 3D environmental data frame and corresponds to the same field of view in space.

[0033] Preliminary obstacle detection and segmentation are performed based on point cloud data. Clustering-based methods (such as Euclidean distance clustering, DBSCAN algorithm, etc.) can be used to group spatially adjacent point clouds into independent point cloud clusters, each of which is considered to potentially correspond to an obstacle. For each obstacle point cloud cluster, the spatial distance between it and the UAV body is calculated. For example, the Euclidean distance between all points in the obstacle point cloud cluster and a preset reference point on the UAV (such as the geometric center of the fuselage) is determined, and the minimum value is selected as the first distance between the obstacle and the UAV. The first distance corresponds to the distance between the UAV and the nearest point on the obstacle surface. An obstacle list can be set up, recording the point cloud cluster identifier of each detected obstacle and its corresponding real-time calculated first distance. This list is continuously or periodically updated to monitor the nearest distance between the UAV and all potential obstacles in the vicinity.

[0034] S202. In response to the first distance being less than or equal to the first distance threshold, the environmental image is input into the image description model to obtain the description text for the environmental image, and the description text is input into the risk assessment model to obtain the risk assessment text for the obstacle.

[0035] In practice, the first distance is compared with a preset first distance threshold. The first distance threshold can be set to 10 meters. Alternatively, the first distance threshold can be set based on the drone's size, current flight speed, braking performance, and safety redundancy requirements. For example, in low-speed scenarios (such as flight speeds below 5 m / s), the first distance threshold can be set to 10 meters; in high-speed scenarios (such as flight speeds above 5 m / s), the first distance threshold can be set to 20 meters or higher. When the first distance is greater than the first distance threshold, it is determined that the obstacle does not pose an imminent threat in space, and the drone maintains its normal flight state while continuously monitoring. When the first distance is less than or equal to the first distance threshold, a risk warning needs to be issued.

[0036] To improve the processing efficiency and relevance of subsequent image description models, preprocessing can be performed before inputting the environmental image into the model. This involves projecting the 3D point cloud clusters corresponding to the obstacle to be evaluated onto the 2D pixel plane of the synchronously acquired environmental image, based on sensor calibration parameters and coordinate transformation relationships. By calculating the pixel bounding box of this point cloud cluster in the image, the image region primarily containing the obstacle can be highlighted in the environmental image (e.g., by increasing the brightness of this region and labeling it with the text "obstacle"). This eliminates interference from irrelevant backgrounds in the image, allowing the model to focus its attention on the obstacle itself.

[0037] Environmental images are input into a pre-trained image description model. This model, an AI based on deep learning (such as an encoder-decoder architecture or a multimodal large model based on visual Transformers), transforms the input visual content into natural language descriptions. The image description model understands and analyzes the input environmental images, outputting one or more descriptive texts. These descriptions not only identify the categories of obstacles (such as "trees," "power lines," "construction cranes," and "pedestrians"), but may also include their states, attributes, and details of their interaction with the scene. For example, for different obstacles, descriptions might include: "A tall poplar tree, with dense foliage, is being blown by the wind, its branches swaying violently," "A black power line crossing the road, with an insulating layer on its surface, reflecting sunlight," "A stationary metal lamppost with an advertising sign on its surface," and "A delivery scooter being ridden, its trajectory unstable." For multiple consecutive environmental images, one image can be selected as the image to be described, while the other frames can serve as reference images. The image description model can then more accurately analyze the motion state of obstacles, such as swaying branches, based on the differences between the images. For a single frame of environmental image, the image description model can still analyze the motion state of the obstacle from its pose. For example, the motion state of a garbage bag floating in the air is necessarily non-static and unpredictable.

[0038] After obtaining the descriptive text, it is combined with a preset prompt word template to form a complete model prompt word, which is then input into the risk assessment model. The risk assessment model can be a generative artificial intelligence model (i.e., a large language model, such as DeepSeek), which possesses powerful natural language understanding, reasoning, and generation capabilities. The prompt word template is a pre-designed text structure used to guide the model to perform specific tasks, such as instructing the generative artificial intelligence model to assess the risk of the obstacle based on the descriptive text, thereby generating risk assessment text. For example: "You are a drone flight safety assessment expert. Based on the following description of the obstacle ahead, assess the risk of a collision with the drone, and output the assessment results strictly according to the following format:" [Risk Level]: Collision Risk [High / Medium-High / Medium-Low / Low] [Main Basis]: Fill the corresponding positions in the template with the descriptive text. For example, for the description "A tall poplar tree, with dense foliage, is being blown by the wind, its branches swaying violently," the complete prompt would be: "You are a drone flight safety assessment expert. Based on the following description of the obstacle ahead, assess the risk of a collision with the drone, and output the assessment results strictly according to the following format:" [Obstacle Description]: A tall poplar tree with lush foliage is being blown by the wind, and its branches are swaying violently.

[0039] [Risk Level]: Collision Risk [High / Medium-High / Medium-Low / Low] [Main Basis]: The prompt is then input into the risk assessment model. Based on its built-in knowledge of drone flight principles, physical laws, and contextual understanding, the model parses and reasons about the descriptive text, generating a structured risk assessment text. For the example above, the risk assessment model might output: "

Risk Level

[0040] [Main Basis]: The obstacle is a tree, whose branches and leaves are in a non-static and unpredictable state of motion under the influence of wind. The large swaying of the branches greatly increases the uncertainty of the drone's passable space, making accidental collisions more likely. S203. In response to the risk assessment text indicating that the obstacle poses a collision risk to the drone, control the drone to avoid the obstacle.

[0041] In specific implementation, the risk level corresponding to the obstacle is determined based on the risk assessment text; when the risk level is greater than or equal to the risk level threshold, it is determined that the obstacle poses a collision risk to the drone; the drone is then controlled to avoid the obstacle based on the risk level. Specifically, the text content is analyzed according to preset rules (such as keyword matching, regular expressions, or lightweight natural language understanding models). For example, the word "high" is extracted from the field "[Risk Level]: High Collision Risk," or direct statements such as "High Collision Risk" are identified. The extracted quantitative risk information (such as high / medium-high / medium-low / low levels) or qualitative conclusions are compared with the preset risk level threshold. The risk level threshold is set to correspond to "medium-low." When the parsed risk level is high / medium-high / medium-low, it is determined that "there is a collision risk"; if it is "low," it is determined that "the risk is acceptable," and emergency obstacle avoidance is not initiated, but it can be recorded and monitored. If the determination result is that the obstacle poses a collision risk to the drone, the drone is controlled to avoid the obstacle.

[0042] In this embodiment, semantic analysis and reasoning of obstacles are performed through image description models and risk assessment models. This transcends the single distance dimension, identifying the category, state (such as moving, stationary, or swaying), and potential threat characteristics of obstacles. This allows the drone to more accurately assess the actual risk, effectively avoiding misjudgments or omissions caused by the inability of traditional methods to distinguish between "a tree branch swaying in the wind" and "a stationary billboard." Furthermore, by controlling the drone to avoid obstacles based on the risk level indicated by the risk assessment text, a shift from "indiscriminate emergency obstacle avoidance" to "risk-level response" is achieved. For low-risk obstacles, only careful monitoring or slight adjustments to the flight path are required, while for high-risk obstacles, decisive avoidance actions are triggered. This differentiated response strategy minimizes unnecessary hovering or detouring while ensuring safety, guaranteeing the smoothness and efficiency of drone operations.

[0043] In some embodiments, controlling the drone to avoid the obstacle based on the risk level includes: The system acquires the current flight status parameters of the UAV; determines a target avoidance strategy based on the risk level and a specified avoidance strategy library, wherein the specified avoidance strategy library includes avoidance strategies corresponding to different risk levels, and the avoidance strategy includes at least one of an emergency braking strategy, a path replanning strategy, and an altitude adjustment strategy; generates flight control commands based on the target avoidance strategy and the current flight status parameters, and controls the UAV to avoid the obstacle based on the flight control commands.

[0044] Once a collision risk is determined, the current flight status parameters of the drone are obtained, including but not limited to the following parameters: the drone's three-dimensional position, three-dimensional velocity vector, attitude angle, real-time relative distance (first distance) and relative orientation with the target obstacle, current flight mode, preset trajectory information and remaining battery power.

[0045] The designated avoidance strategy library establishes mapping rules between different risk levels, different flight states, and specific avoidance strategies. Avoidance strategies mainly include, but are not limited to: emergency braking strategy (reducing forward momentum and hovering), path replanning strategy (recalculating a new path around the current obstacle), and altitude adjustment strategy (passing over or under the obstacle by vertical climb or descent).

[0046] The strategy library can be queried based on the risk level. For example, for "risk level: high", the strategy library may map to "emergency braking strategy"; for "risk level: medium-high", it may map to "path replanning strategy"; and for "risk level: medium-low", it may map to "height adjustment strategy".

[0047] After determining the target avoidance strategy, specific and executable flight control commands are determined based on the target avoidance strategy, the current flight state parameters, and the three-dimensional data (point cloud data) of the obstacle. For example, if the target avoidance strategy is "climb upwards to overcome the obstacle," and the three-dimensional data of the obstacle shows an obstacle height of 20 meters, while the current flight state parameters show the drone's altitude as 10 meters, then the drone must climb upwards until its altitude exceeds the obstacle's height to overcome it. Therefore, the control system calculates a smooth vertical climb trajectory based on the current altitude, attitude, and power performance, and decomposes it into a series of high-frequency attitude and throttle flight control commands. The flight control commands are sent to the drone's power and servo mechanisms in real time for execution, driving the drone to complete corresponding avoidance maneuvers, such as deceleration and hovering, circling, or climbing / descending, thereby effectively avoiding the obstacle. During execution, the obstacle's state and the drone's own state are continuously monitored until the collision risk is eliminated.

[0048] In this embodiment, by acquiring the current flight status parameters of the UAV and querying a specified avoidance strategy library based on the risk level, the abstract risk level is mapped to a specific target avoidance strategy, such as an emergency braking strategy, a path replanning strategy, or an altitude adjustment strategy. This, combined with the flight status parameters, generates precise flight control commands. Thus, obstacle avoidance response is no longer a single or fixed action, but can adaptively adjust according to the actual threat level of the obstacle and the UAV's real-time status. This significantly improves the accuracy and efficiency of obstacle avoidance decisions.

[0049] In some embodiments, the method further includes: The flight speed of the drone is obtained. When the flight speed is greater than or equal to a specified flight speed (e.g., 10 m / s), a first adjustment coefficient is determined based on a first relative difference between the flight speed and the specified flight speed to adjust a first distance threshold. The first relative difference is positively correlated with the first adjustment coefficient, and the first adjustment coefficient is greater than 1. The product of the first distance threshold and the first adjustment coefficient is used as the adjusted first distance threshold. First relative difference = (flight speed - specified flight speed) / specified flight speed. As shown in Table 1: Table 1

[0050] Table 1 illustrates the positive correlation between the first relative difference and the first adjustment coefficient. The larger the first relative difference, the larger the first adjustment coefficient.

[0051] The method further includes: dividing the detection space in front of the UAV (usually determined by LiDAR) into a series of regular three-dimensional voxel grids. The number of points in each non-empty voxel (i.e., a grid cell containing at least one point cloud data) is counted, and the average number of points across all non-empty voxels is calculated. A lower average number of points indicates a more dispersed and sparse point cloud distribution in space, suggesting a potentially more "open" environment; a higher average number of points indicates a denser point cloud distribution, potentially signifying complex obstacle surfaces or a large number of small objects in the environment, indicating high environmental clutter. When the average number of points is greater than or equal to a specified average number of points (e.g., 3), a second adjustment coefficient is determined based on a second relative difference between the average number of points and the specified average number of points. The second relative difference is positively correlated with the second adjustment coefficient, and the second adjustment coefficient is greater than 1. The product of the first distance threshold and the second adjustment coefficient is used as the adjusted first distance threshold. Second relative difference = (average number of points - specified average number of points) / specified average number of points. As shown in Table 2: Table 2

[0052] Table 2 illustrates the positive correlation between the second relative difference and the second adjustment coefficient; the larger the second relative difference, the larger the second adjustment coefficient.

[0053] In this embodiment, when the flight speed is greater than or equal to a specified flight speed, the first distance threshold is increased based on a positive correlation between the flight speed and the specified flight speed. This allows for earlier triggering of risk assessment and obstacle avoidance decisions under high-speed flight conditions, providing the drone with a longer braking distance and reaction time, effectively avoiding the risk of obstacle avoidance delay or failure due to excessive speed. Simultaneously, by dividing the detection space in front of the drone into a three-dimensional voxel grid and calculating the average number of points, the first distance threshold is increased based on a positive correlation between the average number of points and a specified average number of points. This provides early warning of potential obstacles in scenarios with high point cloud density (i.e., high environmental clutter), enhancing the ability to identify and tolerate noise and small objects in complex or dense environments, and reducing missed detections caused by environmental interference. This enables the drone to more intelligently balance obstacle avoidance sensitivity and flight efficiency while ensuring safety.

[0054] All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.

[0055] Based on the same inventive concept, corresponding to the drone obstacle avoidance method provided in the embodiments of this application, this application also provides a drone obstacle avoidance device.

[0056] refer to Figure 3 The drone obstacle avoidance device includes: The acquisition module 301 is configured to acquire environmental images and three-dimensional environmental data collected by the UAV, and determine a first distance between the UAV and the obstacle based on the three-dimensional environmental data. The assessment module 302 is configured to, in response to the first distance being less than or equal to a first distance threshold, input an environmental image into an image description model to obtain a description text for the environmental image, and input the description text into a risk assessment model to obtain a risk assessment text for the obstacle; Control module 303 is configured to control the drone to avoid the obstacle in response to the risk assessment text indicating that the obstacle poses a collision risk to the drone.

[0057] In one possible implementation, the risk assessment model includes a generative artificial intelligence model.

[0058] In another possible implementation, the evaluation module 302 is used for: The descriptive text is inserted into the prompt word template to obtain model prompt words. The prompt word template is used to instruct the generative artificial intelligence model to assess the risk of the obstacle based on the descriptive text in order to generate risk assessment text. The model prompts are input into a generative artificial intelligence model to obtain a risk assessment text for the obstacle.

[0059] In another possible implementation, the control module 303 is used for: The risk level corresponding to the obstacle is determined based on the risk assessment text; When the risk level is greater than or equal to the risk level threshold, it is determined that the obstacle poses a collision risk to the drone; The drone is controlled to avoid the obstacle based on the risk level.

[0060] In another possible implementation, the control module 303 is used for: Obtain the current flight status parameters of the UAV; Based on the risk level and the designated avoidance strategy library, a target avoidance strategy is determined, wherein the designated avoidance strategy library includes avoidance strategies corresponding to different risk levels, and the avoidance strategy includes at least one of emergency braking strategy, path replanning strategy and altitude adjustment strategy; Based on the target avoidance strategy and the current flight status parameters, flight control commands are generated, and the UAV is controlled to avoid the obstacle based on the flight control commands.

[0061] In another possible implementation, the environmental 3D data includes point cloud data; the acquisition module 301 is used for: The geometric boundaries of the obstacle are determined based on the point cloud data; Determine the minimum Euclidean distance between the UAV and the geometric boundary of the obstacle, and use the minimum Euclidean distance as the first distance.

[0062] It should be noted that the obstacle avoidance device for drones provided in the above embodiments is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the drone obstacle avoidance device and the drone obstacle avoidance method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0063] Based on the same inventive concept, corresponding to the drone obstacle avoidance method provided in the embodiments of this application, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the drone obstacle avoidance method described in the above embodiments.

[0064] Figure 4This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0065] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0066] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0067] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0068] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0069] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0070] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0071] The electronic devices described above are used to implement the corresponding UAV obstacle avoidance methods in the foregoing embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0072] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including instructions that can be executed by a processor in a terminal to complete the drone obstacle avoidance method described above. This computer-readable storage medium can be non-transitory. For example, the computer-readable storage medium can be ROM (Read-Only Memory), RAM (Random Access Memory), CD-ROM (Compact Disc Read-Only Memory), magnetic tape, floppy disk, and optical data storage devices, etc.

[0073] In an exemplary embodiment, a computer program product is also provided, including computer program instructions that, when executed on a computer, cause the computer to perform the drone obstacle avoidance method described in the above embodiments.

[0074] It should be noted that the information (including but not limited to user equipment information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals (including but not limited to signals transmitted between user terminals and other devices, etc.) involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0075] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0076] It should be understood that "multiple" as used herein refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, the step numbers described herein are merely illustrative of one possible execution order. In some other embodiments, the steps may not be executed in numerical order, such as two steps with different numbers being executed simultaneously, or two steps with different numbers being executed in the reverse order of the illustration. This application does not limit this.

[0077] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for obstacle avoidance by unmanned aerial vehicles (UAVs), characterized in that, include: The system acquires environmental images and 3D environmental data collected by the drone, and determines a first distance between the drone and the obstacle based on the 3D environmental data. In response to the first distance being less than or equal to a first distance threshold, the environmental image is input into an image description model to obtain a description text for the environmental image, and the description text is input into a risk assessment model to obtain a risk assessment text for the obstacle; In response to the risk assessment text indicating that the obstacle poses a collision risk to the drone, the drone is controlled to avoid the obstacle.

2. The obstacle avoidance method for unmanned aerial vehicles according to claim 1, characterized in that, The risk assessment model includes a generative artificial intelligence model.

3. The obstacle avoidance method for unmanned aerial vehicles according to claim 2, characterized in that, The step of inputting the descriptive text into the risk assessment model to obtain the risk assessment text for the obstacle includes: The descriptive text is inserted into the prompt word template to obtain model prompt words. The prompt word template is used to instruct the generative artificial intelligence model to assess the risk of the obstacle based on the descriptive text in order to generate risk assessment text. The model prompts are input into a generative artificial intelligence model to obtain a risk assessment text for the obstacle.

4. The obstacle avoidance method for unmanned aerial vehicles according to claim 1, characterized in that, The response to the risk assessment text indicating a collision risk to the drone from the obstacle, controlling the drone to avoid the obstacle, includes: The risk level corresponding to the obstacle is determined based on the risk assessment text; When the risk level is greater than or equal to the risk level threshold, it is determined that the obstacle poses a collision risk to the drone; The drone is controlled to avoid the obstacle based on the risk level.

5. The obstacle avoidance method for unmanned aerial vehicles according to claim 4, characterized in that, The method of controlling the drone to avoid the obstacle based on the risk level includes: Obtain the current flight status parameters of the UAV; Based on the risk level and the designated avoidance strategy library, a target avoidance strategy is determined, wherein the designated avoidance strategy library includes avoidance strategies corresponding to different risk levels, and the avoidance strategy includes at least one of emergency braking strategy, path replanning strategy and altitude adjustment strategy; Based on the target avoidance strategy and the current flight status parameters, flight control commands are generated, and the UAV is controlled to avoid the obstacle based on the flight control commands.

6. The obstacle avoidance method for unmanned aerial vehicles according to claim 1, characterized in that, The environmental 3D data includes point cloud data; determining the first distance between the UAV and the obstacle based on the environmental 3D data includes: The geometric boundaries of the obstacle are determined based on the point cloud data; Determine the minimum Euclidean distance between the UAV and the geometric boundary of the obstacle, and use the minimum Euclidean distance as the first distance.

7. An obstacle avoidance device for unmanned aerial vehicles (UAVs), characterized in that, include: The acquisition module is configured to acquire environmental images and three-dimensional environmental data collected by the UAV, and determine a first distance between the UAV and the obstacle based on the three-dimensional environmental data. An assessment module is configured to, in response to the first distance being less than or equal to a first distance threshold, input an environmental image into an image description model to obtain a description text for the environmental image, and input the description text into a risk assessment model to obtain a risk assessment text for the obstacle; A control module is configured to control the drone to avoid the obstacle in response to the risk assessment text indicating that the obstacle poses a collision risk to the drone.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method described in any one of claims 1 to 6.

10. A computer program product comprising computer program instructions, characterized in that, When the computer program instructions are executed on a computer, the computer causes the computer to perform the method as described in any one of claims 1 to 6.

Images