Differentiated flight path planning method and device, equipment and storage medium

Abstract

The invention discloses a differentiated flight path planning method, device and equipment and a storage medium, and the method comprises the steps: obtaining flight parameters of an unmanned plane executing a flight task in an indoor environment, and obtaining a three-dimensional semantic scene graph of the indoor environment, and the three-dimensional semantic scene graph comprises a plurality of voxels; obtaining semantic information of each voxel, wherein the semantic information comprises a position vector of the voxel in the indoor environment and a semantic tag of the voxel; according to a preset semantic tag-cost weight mapping relation, respectively obtaining a cost weight corresponding to the semantic tag of each voxel; and according to the semantic label, the cost weight and the position vector of each voxel in the three-dimensional semantic scene graph, performing path planning on a flight path from the flight starting point to the flight end point to obtain a differentiated flight path of the unmanned aerial vehicle. According to the method, the unmanned aerial vehicle can plan a more reasonable, safer and more efficient flight path in an indoor complex environment based on understanding of semantic information of the environment.

Description

Technical Field

[0001] The present invention relates to the field of computer indoor positioning technology, and in particular to a differentiated flight path planning method, device, equipment and storage medium. Background Art

[0002] With the continuous advancement of technology, drones have gained widespread application in numerous fields, such as surveillance, logistics, and disaster relief, thanks to their significant advantages, including high maneuverability, efficient mission execution, and relatively low cost. However, when drones are used in indoor environments, traditional navigation and path planning methods face significant challenges. Indoor spaces are often limited, significantly reducing the available flight area for drones compared to open outdoor environments. This makes drones highly susceptible to collisions with surrounding objects during flight. Furthermore, dynamic obstacles, such as moving people and equipment, can exist indoors. These dynamic elements increase environmental uncertainty, thereby reducing the reliability of traditional navigation and path planning methods. Summary of the Invention

[0003] Based on this, it is necessary to propose a differentiated flight path planning method, device, equipment and storage medium to address the problem that the existing drone flight path planning method is difficult to adapt to complex indoor spaces.

[0004] A first aspect of the present invention provides a differentiated flight path planning method, which is applied to a drone. The differentiated flight path planning method includes:

[0005] Obtaining flight parameters of a UAV performing a flight mission in an indoor environment, the flight parameters including a flight start point and a flight end point, and obtaining a three-dimensional semantic scene graph of the indoor environment, wherein the three-dimensional semantic scene graph includes a number of voxels;

[0006] Acquire semantic information of each voxel, where the semantic information includes a position vector of the voxel in the indoor environment and a semantic label of the voxel;

[0007] According to a preset semantic label-cost weight mapping relationship, respectively obtaining the cost weight corresponding to the semantic label of each voxel;

[0008] According to the semantic label, cost weight and position vector of each voxel in the three-dimensional semantic scene graph, a flight path from the flight start point to the flight end point is planned to obtain a differentiated flight path of the UAV.

[0009] Furthermore, the cost weights include at least a high cost weight and a low cost weight, wherein the flight obstacle corresponding to the high cost weight is greater than the flight obstacle corresponding to the low cost weight;

[0010] The step of performing path planning on a flight path from the flight start point to the flight end point based on the semantic label, cost weight, and position vector of each voxel in the three-dimensional semantic scene graph to obtain a differentiated flight path for the UAV includes:

[0011] performing path planning on a flight path from the flight start point to the flight end point based on a B-spline curve according to the cost weight and position vector of each voxel in the three-dimensional semantic scene graph, so that the flight path preferentially selects voxels passing through low-cost weights, wherein the B-spline curve includes a plurality of control points;

[0012] For each control point on the B-spline curve, a corresponding obstacle anchor point is assigned on the three-dimensional semantic scene graph, and a semantic label of a voxel where each obstacle anchor point is located is obtained as an anchor point semantic label;

[0013] According to the anchor point semantic label corresponding to each control point and the cost weight corresponding to the anchor point semantic label, the position of the corresponding control point is adjusted to obtain an adjusted B-spline curve;

[0014] According to the anchor point semantic label corresponding to each control point, differentiated flight speed parameters are assigned to the adjusted B-spline curve to obtain the differentiated flight path of the UAV.

[0015] Furthermore, the step of adjusting the position of each control point according to the anchor point semantic label corresponding to each control point and the cost weight corresponding to the anchor point semantic label to obtain the adjusted B-spline curve includes:

[0016] According to the preset semantic label-cost weight mapping relationship, each control point Q is obtained i Corresponding cost weight λ(S), i=1, 2...N c ;

[0017] According to the formula S λ(S) =S f-orin ·λ(S), calculate each control point Q separately i The corresponding obstacle avoidance gap S λ(S) , where S f-orin is the preset global safety gap;

[0018] Adjust each control point Q separately i and the corresponding anchor point P ij The distance from the control point Q i The corresponding obstacle avoidance gap S λ(S) , j is the index of the {p, v} pair, where p represents the anchor point and v represents the repulsive force direction vector associated with the anchor point;

[0019] The B-spline curve after all control points are adjusted is used as the adjusted B-spline curve.

[0020] Furthermore, the step of assigning differentiated flight speed parameters to the adjusted B-spline curve according to the anchor point semantic label corresponding to each control point to obtain the differentiated flight path of the UAV includes:

[0021] The adjusted B-spline curve is parameterized, and each control point in the adjusted B-spline curve is represented as {Q1, Q2, Q3, ..., Q Nc}, set the time interval Δt between adjacent control points to be equal, according to the formula and Calculate the initial velocity parameters of the i-th control point, which include the initial velocity V i , acceleration A i , acceleration J i ;

[0022] The objective function of speed parameter optimization is defined as J = λ s J smooth +λ c J collision +λ d J dynamic , where J smooth For smoothing penalty, J collision For collision penalty, J dynamic is the dynamic feasibility penalty, parameter λ s is the weight of the smoothing penalty, parameter λ c is the weight of the collision penalty, parameter λ d is the weight of dynamic feasibility penalty;

[0023] For each control point Q i The initial speed parameters are optimized to minimize the objective function Get each control point Q i Optimized speed parameters;

[0024] After completing the speed parameter optimization at all control points, the differentiated flight path of the UAV is obtained.

[0025] Furthermore, the smoothing penalty is expressed as:

[0026] The collision penalty is expressed as:

[0027]

[0028] Among them, Nc represents the total number of control points, and Np represents the number of control points corresponding to Q. iThe number of {p, v} pairs, i represents the control point index, j represents the index of the {p, v} pair, c ij =S λ(S) -d ij , S λ(S) For each control point Q i The corresponding obstacle avoidance gap, d ij is the control point Q i and the corresponding anchor point P ij The actual distance;

[0029] The dynamic feasibility penalty is expressed as: J dynamic =∑ i＝1 w v F(V i )+∑ i＝1 w a F(A i )+∑ i＝1 w j F(J i ), where wv, wa, w j is the weight coefficient, and F is the quadratic continuous differentiable function of the higher-order derivative at the i-th control point.

[0030] Furthermore, the objective function for speed parameter optimization is defined as J=λ s J smooth +λ c J collision +λ d J dynamic The steps include:

[0031] Determine whether the anchor semantic label corresponding to the i-th control point is a preset high-cost weight;

[0032] If so, the dynamic feasibility penalty is expressed as:

[0033] v} pairs, V ij is the direction vector.

[0034] Furthermore, before the step of obtaining the three-dimensional semantic scene graph of the indoor environment, the method further includes:

[0035] Acquire a two-dimensional image and a depth image of the indoor environment, perform semantic segmentation on the two-dimensional image to obtain two-dimensional semantic features, and perform depth estimation on the two-dimensional image to obtain a depth estimation image;

[0036] Performing depth completion on the depth image according to the depth estimation image to obtain a completed depth image, and extracting depth features in the completed depth image;

[0037] Under the guidance of the depth feature, the two-dimensional semantic feature is mapped to the three-dimensional space, and the two-dimensional semantic feature is associated with the corresponding depth feature to obtain a three-dimensional structural feature and a three-dimensional semantic feature;

[0038] The three-dimensional structural features and the three-dimensional semantic features are feature-fused to obtain a three-dimensional semantic scene graph, wherein each voxel in the three-dimensional semantic scene graph carries three-dimensional semantic information converted from the three-dimensional semantic features corresponding to the voxel.

[0039] A second aspect of the present invention provides a differentiated flight path planning device, which is applied to a drone. The differentiated flight path planning device includes:

[0040] a data acquisition module, configured to acquire flight parameters of a UAV performing a flight mission in an indoor environment, the flight parameters including a flight start point and a flight end point, and to acquire a three-dimensional semantic scene graph of the indoor environment, wherein the three-dimensional semantic scene graph includes a number of voxels;

[0041] A semantic acquisition module, configured to acquire semantic information of each voxel, wherein the semantic information includes a position vector of the voxel in the indoor environment and a semantic label of the voxel;

[0042] A cost weight acquisition module is used to obtain the cost weight corresponding to the semantic label of each voxel according to a preset semantic label-cost weight mapping relationship;

[0043] The path planning module is used to plan the flight path from the flight starting point to the flight end point according to the semantic label, cost weight and position vector of each voxel in the three-dimensional semantic scene graph, so as to obtain a differentiated flight path of the UAV.

[0044] A third aspect of the present invention provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of the above-mentioned differentiated flight path planning method.

[0045] A fourth aspect of the present invention provides a device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above-mentioned differentiated flight path planning method.

[0046] The differentiated flight path planning method, device, equipment, and storage medium of the present invention utilizes drone flight parameters and a three-dimensional semantic scene graph of the indoor environment, combining voxel semantic information with cost weights for path planning. This enables drones to plan more reasonable, safe, and efficient flight paths in complex indoor environments based on their understanding of the semantic information of the environment. Compared to traditional path planning methods, this method fully considers the semantic attributes of objects in the indoor environment, avoids entering high-risk areas, and improves the drone's ability to perform missions in complex indoor environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0048] in:

[0049] Figure 1 FIG. 1 is a diagram illustrating an application environment of a differentiated flight path planning method according to an embodiment;

[0050] Figure 2 is a flowchart of a differentiated flight path planning method in one embodiment;

[0051] Figure 3 Schematic diagram of control points and obstacle anchor points in a flight trajectory in one embodiment, where (a) is a schematic diagram of {p, v} pairs; (b) is a schematic diagram of the repulsive force direction and obstacle avoidance gap in {p, v};

[0052] Figure 4 is a structural block diagram of a differentiated flight path planning device in one embodiment;

[0053] Figure 5 FIG. 1 is a structural block diagram of a computer device in one embodiment. DETAILED DESCRIPTION

[0054] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0055] Figure 1 FIG. 1 is a diagram of an application environment for differentiated flight path planning in an embodiment. Figure 1The differentiated flight path planning method is applied to a differentiated flight path planning system. The differentiated flight path planning system includes a terminal 110 and a server 120. Terminal 110 and server 120 are connected via a network. Terminal 110 can be a desktop terminal or a mobile terminal. The mobile terminal can be at least one of a mobile phone, a tablet computer, a laptop computer, etc. Server 120 can be implemented as an independent server or a server cluster consisting of multiple servers. Terminal 110 is used to execute the steps of the differentiated flight path planning method to plan an indoor flight path for the drone, and server 120 is used to store a three-dimensional semantic scene graph of the indoor environment.

[0056] like Figure 2 As shown, in one embodiment, a differentiated flight path planning method is provided, which is applied to a UAV and includes the following steps:

[0057] S1: Obtain flight parameters of a UAV performing a flight mission in an indoor environment, wherein the flight parameters include a flight start point and a flight end point, and obtain a three-dimensional semantic scene graph of the indoor environment, wherein the three-dimensional semantic scene graph includes a number of voxels;

[0058] S2: Acquire semantic information of each voxel, where the semantic information includes a position vector of the voxel in the indoor environment and a semantic label of the voxel;

[0059] S3: Obtaining the cost weight corresponding to the semantic label of each voxel according to a preset semantic label-cost weight mapping relationship;

[0060] S4: According to the semantic label, cost weight and position vector of each voxel in the three-dimensional semantic scene graph, a flight path from the flight start point to the flight end point is planned to obtain a differentiated flight path of the UAV.

[0061] In this embodiment, in the above step S1, key flight parameters such as the flight start point and flight end point are extracted from the flight mission instructions of the UAV. The above three-dimensional semantic scene graph can be pre-constructed, or the indoor environment can be scanned and data collected in real time by sensors (such as lidar, camera, etc.), and a three-dimensional semantic scene graph is constructed through semantic segmentation, image completion, feature fusion and other processing. The three-dimensional semantic scene graph carries multiple semantic labels, and the semantic labels are used to mark the category names of items in the three-dimensional semantic scene graph. The semantic labels include, for example, "wall", "bed", "door", "window", "passageway", "ground" and so on. The three-dimensional semantic scene graph is discretized into a number of voxels, each voxel representing a small unit in space. Each voxel is represented by the following components: a position vector x i ∈R 3 , semantic tags i ∈S, occupancy probability pocc (vi)∈[0,1].

[0062] In step S2 above, for each voxel in the three-dimensional semantic scene graph, the precise position vector of the voxel in the indoor environment is determined, and the semantic label corresponding to the position vector of each voxel is obtained. This allows the drone to distinguish different types of spatial areas and objects, understand the properties of each part of the environment, and facilitate subsequent differentiated processing based on different semantics.

[0063] In the above step S3, the semantic label-cost weight mapping relationship is pre-set. Through empirical analysis or data-driven learning, the cost weight distribution can be further refined to improve the efficiency of flight path planning. The cost weight is defined as λ(s), where:

[0064]

[0065] The higher the λ(s), the higher the collision cost of the area corresponding to the category, and it should be avoided; the lower the λ(s), the lower the collision cost of the area, and navigation should be prioritized. The cost weights are in the order of λ from high to low. danger >λ obstacle >λ neutral >λ priority For example, the semantic label "Wall" corresponds to a higher cost weight, while the semantic label "Door" corresponds to a lower cost weight. Based on this mapping relationship, the semantic label of each voxel is matched with a corresponding cost weight. By quantifying the cost weights of voxels with different semantic labels in path planning, path planning can be differentiated based on the collision costs of different voxels.

[0066] In step S4, the semantic label, cost weight, and position vector of each voxel in the three-dimensional semantic scene graph are integrated to apply a path planning algorithm (such as the EGO-planner algorithm). When searching for a path from the flight start point to the flight end point, the cost weight of the voxels is taken into account, and paths passing through voxels with low cost weights are preferred, while voxels with high cost weights are avoided. This allows for the planning of a differentiated flight path, generating a flight path that conforms to the semantic characteristics of the indoor environment. This allows the drone to safely and efficiently fly from the start point to the end point while taking into account the semantic attributes of different objects and spaces, reducing collision risks and improving flight efficiency.

[0067] The differentiated flight path planning method of this embodiment acquires drone flight parameters and a three-dimensional semantic scene graph of the indoor environment, then combines voxel semantic information with cost weights for path planning. This enables drones to plan more reasonable, safe, and efficient flight paths in complex indoor environments based on their understanding of the environment's semantic information. Compared to traditional path planning methods, this method fully considers the semantic attributes of objects in the indoor environment, avoiding high-risk areas and improving the drone's ability to perform missions in complex indoor environments.

[0068] In a specific embodiment, the cost weight includes at least a high cost weight and a low cost weight, wherein the flight obstacle corresponding to the high cost weight is greater than the flight obstacle corresponding to the low cost weight;

[0069] The step S4 of performing path planning on a flight path from the flight start point to the flight end point based on the semantic label, cost weight, and position vector of each voxel in the three-dimensional semantic scene graph to obtain a differentiated flight path for the UAV includes:

[0070] S41: performing path planning on a flight path from the flight start point to the flight end point based on a B-spline curve according to the cost weight and position vector of each voxel in the three-dimensional semantic scene graph, so that the flight path preferentially selects voxels passing through low-cost weights, wherein the B-spline curve includes a plurality of control points;

[0071] S42: assigning a corresponding obstacle anchor point to each control point on the B-spline curve on the three-dimensional semantic scene graph, and obtaining a semantic label of a voxel where each obstacle anchor point is located as an anchor point semantic label;

[0072] S43: adjusting the position of each control point according to the anchor point semantic label corresponding to each control point and the cost weight corresponding to the anchor point semantic label to obtain an adjusted B-spline curve;

[0073] S44: According to the anchor point semantic label corresponding to each control point, a differentiated flight speed parameter is assigned to the adjusted B-spline curve to obtain a differentiated flight path of the UAV.

[0074] In this embodiment, in step S41, the EGO-planner algorithm performs path planning based on a B-spline curve. The control points of the B-spline curve are initialized. The initial positions of these control points can be set based on the flight start and end points and the general layout of the scene. For example, a certain number of control points can be evenly distributed between the start and end points. A path objective function is defined, which includes a penalty term for passing through voxels with high cost weights. This ensures that during the path planning process, the algorithm tends to make the B-spline curve (i.e., the planned flight path) pass through voxels with low cost weights more frequently. For example, if one voxel represents a wall (high cost weight) and another represents a passage (low cost weight), the algorithm will try to make the flight path pass through the voxels corresponding to the passage. An optimization algorithm (such as gradient descent) is used to iteratively optimize the path objective function, continuously adjusting the positions of the B-spline curve control points. This gradually optimizes the flight path to prioritize passing through voxels with low cost weights. In each iteration, the relationship between the flight path and the voxel is calculated based on the voxel's position vector, and the control point positions are then updated. By setting high and low cost weights for each voxel in the three-dimensional semantic scene graph, path planning prioritizes passing through low-cost voxels and automatically avoids areas with high-cost voxels such as walls and large equipment to reduce collision risks and ensure flight safety. In addition, B-spline curves are used to plan paths based on voxel cost weights and position vectors, avoiding unnecessary path exploration in high-cost areas, reducing calculation volume and search space, enabling drones to find feasible paths from the starting point to the end point more quickly, and improving the speed of path planning.

[0075] In the above step S42, for each control point on the b-spline curve, in the 3D semantic scene graph, with the control point as the center, search for the nearest obstacle (such as a table, bed, wall, etc.) within a certain range. Figure 3 (a) A point on the obstacle closest to a certain repulsive direction (which can be the point closest to the control point) is used as the obstacle anchor point P in the corresponding repulsive direction, and the control point Q i The obstacle distance to the corresponding anchor point is defined as: d ij =(Q i -P ij )v ij , where Q represents the control point, i represents the control point index, and j represents the index of the {p, v} pair, where p represents the anchor point and v represents the repulsive force direction vector associated with the anchor point. Obtain the semantic label of the voxel where the obstacle anchor point is located as the anchor semantic label. For example, if the semantic label of the voxel where the anchor point is located is "table", then the anchor semantic label is "table".

[0076] In step S43, the adjustment amount of the control point position is calculated based on the cost weight of the anchor point semantic label. For example, the greater the cost weight, the greater the amplitude of the control point adjustment in the direction away from the obstacle anchor point. The new position of the control point is determined by means of vector operations and other methods, thereby obtaining the adjusted B-spline curve. For example, when encountering dynamic obstacles such as moving people (specific semantic labels), the path can be adjusted in a timely manner based on the cost weight, and dynamic obstacles can be flexibly avoided, further improving flight safety.

[0077] In step S44, differentiated flight parameters are assigned to the flight path segments corresponding to different semantic labels. For example, semantic labels representing safe, open areas (such as "door") are assigned higher flight speed parameters, while semantic labels representing dangerous or narrow areas (such as "cabinet") are assigned lower flight speed parameters. As the drone flies along the flight path, it will fly at corresponding speed parameters in different path segments, resulting in a differentiated flight path for the drone.

[0078] This embodiment utilizes voxel semantic labels and cost weights to enable the drone to understand the attributes and potential risks of different indoor environmental elements, allowing it to fly flexibly while maintaining a certain safe distance and adapting to complex and diverse indoor environments. In addition, the entire process, from path planning to speed distribution, is dynamically adjusted based on environmental semantic information, allowing the drone to quickly respond appropriately to different indoor layouts, obstacle distribution, and dynamic changes, thereby enhancing its adaptability to various indoor environments.

[0079] Reference Figure 3 (b) In a specific embodiment, the step S43 of adjusting the position of each control point according to the anchor point semantic label corresponding to each control point and the cost weight corresponding to the anchor point semantic label to obtain the adjusted B-spline curve includes:

[0080] S431: According to the preset semantic label-cost weight mapping relationship, each control point Q is obtained. i Corresponding cost weight λ(S), i=1, 2...N c ;

[0081] S432: According to formula S λ(S) =S f-orin ·λ(S), calculate each control point Q separately i The corresponding obstacle avoidance gap S λ(S) , where S f-orin is the preset global safety gap;

[0082] S433: Adjust each control point Q separately i and the corresponding anchor point P ij The distance from the control point Qi The corresponding obstacle avoidance gap S λ(S) , j is the index of the {p, v} pair, where p represents the anchor point and v represents the repulsive force direction vector associated with the anchor point;

[0083] S434: The B-spline curve after all control points are adjusted is used as the adjusted B-spline curve.

[0084] In this embodiment, in the above step S431, for each control point Qi (i = 1, 2, ..., n) on the b-spline curve, according to its corresponding anchor point semantic label, the corresponding cost weight λ(S) is searched and obtained in the pre-set mapping relationship, thereby avoiding the use of a unified obstacle avoidance method for all obstacles, being able to more accurately deal with different types of obstacles, and improving the effect and efficiency of obstacle avoidance.

[0085] In the above step S432, the global safety gap is preset as the minimum safety distance that needs to be maintained between the UAV and the obstacle without considering the specific semantics of the obstacle. f-orin Set to 0.5 meters, indicating that in general, the drone should keep at least 0.5 meters away from obstacles. For each control point Qi, according to the formula S λ(S) =S f-orin ·λ(S) calculates the corresponding obstacle avoidance gap S λ(S) For high-risk obstacles, the cost weight is large, and the calculated obstacle avoidance gap is also large, so the drone can maintain a larger safety distance; for low-risk obstacles, the obstacle avoidance gap is relatively small, which can make more reasonable use of space while ensuring safety.

[0086] In the above step S433, for each control point Q i , and its corresponding anchor point P ij The distance is adjusted to the obstacle avoidance gap S corresponding to the control point λ(S) The repulsive force direction vector v is the control point Q i The adjusted direction is the direction away from obstacles, ensuring that the drone maintains a suitable safe distance from obstacles during flight.

[0087] In the above step S434, all control points Q on the b-spline curve are sequentially i (i=1,2,…,N c ) to adjust the position, and the B-spline curve formed by these adjusted control points is determined to be the adjusted B-spline curve. At this point, this curve can better adapt to the characteristics of different semantic obstacles in indoor environments, ensuring that the drone maintains an appropriate safe distance from obstacles during flight.

[0088] In a specific embodiment, the step S44 of assigning differentiated flight speed parameters to the adjusted B-spline curve according to the anchor point semantic label corresponding to each control point to obtain the differentiated flight path of the UAV includes:

[0089] S441: Parameterize the adjusted B-spline curve, and represent each control point in the adjusted B-spline curve as {Q1, Q2, Q3, ..., Q Nc}, set the time interval Δt between adjacent control points to be equal, according to the formula and Calculate the initial velocity parameters of the i-th control point, which include the initial velocity V i , acceleration A i , acceleration J i ;

[0090] S442: Define the objective function of speed parameter optimization as J=λ s J smooth +λ c J collision +λ d J dynamic , where J smooth For smoothing penalty, J collision For collision penalty, J dynamic is the dynamic feasibility penalty, parameter λ s is the weight of the smoothing penalty, parameter λ c is the weight of the collision penalty, parameter λ d is the weight of dynamic feasibility penalty;

[0091] S443: For each control point Q i The initial speed parameters are optimized to minimize the objective function Get each control point Q i Optimized speed parameters;

[0092] S444: After completing the speed parameter optimization at all control points, the differentiated flight path of the UAV is obtained.

[0093] In this embodiment, in the above step S441, the adjusted b-spline curve is parameterized, and each control point on the curve is represented by a sequence {Q1, Q2, Q3, ..., Q Nc To facilitate subsequent calculations and planning, equal time intervals Δt are set between adjacent control points.

[0094] In the above step S442, the smoothing penalty J smooth Based on the acceleration A i and jerk J iThe purpose of calculation is to make the flight trajectory of the drone smoother, avoid excessive changes in speed and acceleration, and make the flight process more stable and comfortable. For example, if the acceleration is too large, it means that the changes in speed and acceleration are too sudden. smooth The value will increase accordingly.

[0095] Collision Penalty J collision The cost weight λ(s) is weighted based on the different semantic labels of the obstacles. Obstacles with different semantic labels have different collision risks. For example, "walls" have a higher collision risk and a corresponding larger cost weight. This is given a higher weight when calculating the collision penalty to ensure that the drone avoids these obstacles as much as possible.

[0096] Dynamic feasibility penalty J dynamic It is used to ensure that the trajectory follows the dynamic constraints of the UAV, ensuring that the UAV operates within a safe operating range and preventing the execution of inappropriate trajectories.

[0097] Parameter λ s ,λ c and λ d These are the weights of the smoothness penalty, collision penalty, and dynamic feasibility penalty, respectively. They control the importance of each penalty term in the objective function. These weights can be adjusted according to actual needs to balance different optimization objectives.

[0098] In the above steps S443-S444, for each control point Q i Initial velocity parameters (initial velocity V i , acceleration A i , acceleration J i ), use optimization algorithms (such as gradient descent method, etc.) to minimize the objective function J. During the optimization process, the value of the speed parameter is continuously adjusted so that the value of the objective function gradually decreases. For example, when J smooth When the value of is large, the algorithm will try to adjust the velocity parameters to reduce the acceleration, thereby reducing the smoothing penalty; when J collision When the value of is large, the speed and trajectory will be adjusted to avoid obstacles and reduce the risk of collision.

[0099] After multiple iterations of optimization, each control point Q is obtained i The optimized speed parameters can minimize the objective function under the premise of ensuring flight smoothness, avoiding collisions and meeting dynamic feasibility.

[0100] Once the speed parameters of all control points are optimized, the flight speed of each point on the entire adjusted B-spline curve is determined. This not only takes spatial positioning into consideration, but also differentiates the flight speed settings based on the semantic labels of the anchor points corresponding to different control points to adapt to different flight environments.

[0101] In a specific embodiment, the smoothing penalty is expressed as: Without time integration, the trajectory is penalized for squared acceleration and squared jerk.

[0102] The collision penalty is expressed as:

[0103]

[0104] Among them, Nc represents the total number of control points, and Np represents the number of control points corresponding to Q. i The number of {p, v} pairs, i represents the control point index, j represents the index of the {p, v} pair, c ij =S λ(S) -d ij , S λ(S) For each control point Q i The corresponding obstacle avoidance gap, d ij is the control point Q i and the corresponding anchor point P ij The actual distance. c (Q i ) is the control point Q i penalty items.

[0105] When c ij When ≤0, j c (Q i )=0, indicating that the position corresponding to the control point is safe, there is no collision risk, and no collision penalty is generated.

[0106] When 0 <c ij ≤S λ(S) When c ij As increases, the penalty value increases cubically.

[0107] When c ij >S λ(S) When d ij When <0, it means that the control point may be inside the obstacle, which is usually caused by dynamic adjustment or complex obstacle modeling during the optimization process. The penalty value is a ij The quadratic function form is used. Since the local path planning is based on the B-spline curve, the control points on the predicted path are optimized in advance. A control point Q generated by a certain prediction is i Maybe before the control point Q i-1 Under the optimization of i When the error is generated inside the obstacle, a larger penalty is required for optimization. At the same time, it prevents the optimization function from stopping at the obstacle boundary, which would result in an unsafe path, thereby generating a safer path curve.

[0108] The dynamic feasibility penalty is expressed as: J dynamic =∑ i＝1 w v F(V i )+∑ i＝1 w a F(A i )+∑ i＝1 w j F(J i ), where w v , w a , w j is the weight coefficient, and F is a quadratic continuously differentiable function of the higher-order derivative at the i-th control point. The quadratic continuously differentiable function F can be defined based on the allowable range of the UAV's speed, acceleration, and jerk.

[0109] Further, according to Compute the sum of the constraints for each control point, where

[0110] Function f(c r ) is defined as follows:

[0111]

[0112] c r ∈{V i ,A i ,J i}, parameters a1, b1, c1, a2, b2, c2 are selected according to the constraint requirements. Variable c j represents the threshold of the derivative, c m is the boundary limit, and λ is a coefficient related to elasticity, where λ<1 ensures that the result satisfies the constraints. The goal is to minimize the penalty of all constraints and thus further optimize the path planning.

[0113] In a specific embodiment, the objective function defining the speed parameter optimization is J=λ s J smooth +λ c J collision +λ d J dynamic Step S442 includes:

[0114] S4421: Determine whether the anchor semantic label corresponding to the i-th control point is a preset high cost weight;

[0115] S4422: If yes, then the dynamic feasibility penalty is expressed as:

[0116]

[0117] Np represents the control point Qi The number of {p, v} pairs, V ij is the direction vector, that is, the control point Q i To the obstacle anchor point P ij The direction vector of Figure 3 (b) is used to help the optimization algorithm to control the point Q i Push in the correct direction of the anchor point away from obstacles.

[0118] In this embodiment, the control point Q i When located near high-cost obstacles, a higher dynamic feasibility penalty is imposed, which affects the optimization process and ensures that control points near high-cost obstacles receive a larger gradient penalty, thereby guiding the flight path optimization process to avoid these areas. The gradient is calculated by trilinear interpolation on the domain, where J is directly calculated. c Q i The derivative of , Np is the total number of {p,v} pairs.

[0119] In a specific embodiment, before step S1 of obtaining the three-dimensional semantic scene graph of the indoor environment, the following steps are included:

[0120] S01: Acquire a two-dimensional image and a depth image of the indoor environment, perform semantic segmentation on the two-dimensional image to obtain two-dimensional semantic features, and perform depth estimation on the two-dimensional image to obtain a depth estimation image;

[0121] S02: performing depth completion on the depth image according to the depth estimation image to obtain a completed depth image, and extracting depth features in the completed depth image;

[0122] S03: Under the guidance of the depth feature, mapping the two-dimensional semantic feature to a three-dimensional space, and associating the two-dimensional semantic feature with the corresponding depth feature to obtain a three-dimensional structural feature and a three-dimensional semantic feature;

[0123] S04: Fusing the three-dimensional structural features and the three-dimensional semantic features to obtain a three-dimensional semantic scene graph, wherein each voxel in the three-dimensional semantic scene graph carries three-dimensional semantic information converted from the three-dimensional semantic features corresponding to the voxel.

[0124] In this embodiment, in the above step S01, the visual sensors (such as RGB cameras and depth cameras) carried by the drone are used to obtain two-dimensional images (usually RGB images) and depth images of the indoor environment respectively. The two-dimensional image contains rich texture and color information, while the depth image records the distance information from each pixel in the scene to the camera. Semantic segmentation is performed on the two-dimensional image. Semantic segmentation is a technology that classifies each pixel in the image into different semantic categories. For example, the pixels in the image are divided into different categories such as "wall", "ground", "furniture", etc., thereby obtaining two-dimensional semantic features. At the same time, the two-dimensional image is processed by a depth estimation algorithm to predict the depth value of each pixel to obtain a depth estimation image.

[0125] In the above step S02, the depth image may have problems such as noise and missing values, so the depth image is depth-completed based on the depth estimation image. For example, an interpolation algorithm or a deep learning method can be used to use the information in the depth estimation image to fill the missing areas in the depth image to obtain a completed depth image. Depth features are extracted from the completed depth image using methods such as convolutional neural networks. The depth features can reflect the three-dimensional structural information of the scene, such as the shape, size, and relative position of objects. Depth completion improves the quality and integrity of the depth image, making subsequent three-dimensional reconstruction more accurate.

[0126] In step S03, guided by the depth features, the 2D semantic features are mapped to 3D space. The semantic information corresponding to each pixel in the 2D image is converted to its corresponding position in 3D space. The 2D semantic features are associated with the corresponding depth features, for example, by binding each point in 3D space to its corresponding semantic category, thereby obtaining 3D structural features and 3D semantic features. This conversion of 2D semantic information to 3D space enables the drone to understand the semantic information of different objects in 3D space.

[0127] In step S04, the 3D structural features and 3D semantic features are fused into a unified feature representation using methods such as feature concatenation and weighted summation. Based on the fused features, the indoor environment is divided into a number of voxels, and each voxel is assigned 3D semantic information derived from its corresponding 3D semantic features, thereby generating a 3D semantic scene graph. The 3D semantic information carried by each voxel provides rich semantic guidance for subsequent path planning, helping the drone better understand and adapt to the indoor environment.

[0128] This embodiment constructs a 3D semantic scene graph of the indoor environment through the above steps. This 3D semantic scene graph provides more accurate and rich semantic information for drone path planning. Compared to traditional 3D maps based solely on geometric information, it enables drones to better understand the semantic categories and functions of different objects in the indoor environment. The semantic-based path planning method can better adapt to the dynamic changes of the indoor environment and improve the drone's autonomous flight capabilities in complex indoor environments.

[0129] Figure 4 A schematic diagram of a differentiated flight path planning device in one embodiment is shown, which is applied to a drone. The differentiated flight path planning device includes:

[0130] A data acquisition module 10 is configured to acquire flight parameters of a UAV performing a flight mission in an indoor environment, the flight parameters including a flight start point and a flight end point, and to acquire a three-dimensional semantic scene graph of the indoor environment, wherein the three-dimensional semantic scene graph includes a number of voxels;

[0131] A semantic acquisition module 20, configured to acquire semantic information of each voxel, wherein the semantic information includes a position vector of the voxel in the indoor environment and a semantic label of the voxel;

[0132] The cost weight acquisition module 30 is used to respectively acquire the cost weight corresponding to the semantic label of each voxel according to a preset semantic label-cost weight mapping relationship;

[0133] The path planning module 40 is used to plan the flight path from the flight start point to the flight end point according to the semantic label, cost weight and position vector of each voxel in the three-dimensional semantic scene graph, so as to obtain a differentiated flight path of the UAV.

[0134] In a specific embodiment, the cost weight includes at least a high cost weight and a low cost weight, wherein the flight obstacle corresponding to the high cost weight is greater than the flight obstacle corresponding to the low cost weight;

[0135] The path planning module 40 includes:

[0136] a path planning unit, configured to plan a flight path from the flight start point to the flight end point based on a B-spline curve according to the cost weight and position vector of each voxel in the three-dimensional semantic scene graph, so that the flight path preferentially selects voxels passing through low-cost weights, wherein the B-spline curve includes a plurality of control points;

[0137] An anchor point assignment unit, configured to assign a corresponding obstacle anchor point on the three-dimensional semantic scene graph to each control point on the B-spline curve, and obtain a semantic label of a voxel where each obstacle anchor point is located as an anchor point semantic label;

[0138] A first adjustment unit is configured to adjust the position of each control point according to an anchor point semantic label corresponding to each control point and a cost weight corresponding to the anchor point semantic label to obtain an adjusted B-spline curve;

[0139] The second adjustment unit is used to assign differentiated flight speed parameters to the adjusted B-spline curve according to the anchor point semantic label corresponding to each control point, so as to obtain a differentiated flight path of the UAV.

[0140] In some embodiments, the first adjusting unit includes a unit for:

[0141] According to the preset semantic label-cost weight mapping relationship, each control point Q is obtained i Corresponding cost weight λ(S), i=1, 2...N c ;

[0142] According to the formula S λ(S) =S f-orin ·λ(S), calculate each control point Q separately i The corresponding obstacle avoidance gap S λ(S) , where S f-orin is the preset global safety gap;

[0143] Adjust each control point Q separately i and the corresponding anchor point P ij The distance from the control point Q i The corresponding obstacle avoidance gap S λ(S) , j is the index of the {p, v} pair, where p represents the anchor point and v represents the repulsive force direction vector associated with the anchor point;

[0144] The B-spline curve after all control points are adjusted is used as the adjusted B-spline curve.

[0145] In some embodiments, the second adjusting unit includes a unit for:

[0146] The adjusted B-spline curve is parameterized, and each control point in the adjusted B-spline curve is represented as {Q1, Q2, Q3, ..., Q Nc}, set the time interval Δt between adjacent control points to be equal, according to the formula and Calculate the initial velocity parameters of the i-th control point, which include the initial velocity V i , acceleration Ai , acceleration J i ;

[0147] The objective function of speed parameter optimization is defined as J = λ s J smooth +λ c J collision +λ d J dynamic , where J smooth For smoothing penalty, J collision For collision penalty, J dynamic is the dynamic feasibility penalty, parameter λ s is the weight of the smoothing penalty, parameter λ c is the weight of the collision penalty, parameter λ d is the weight of dynamic feasibility penalty;

[0148] For each control point Q i The initial speed parameters are optimized to minimize the objective function Get each control point Q i Optimized speed parameters;

[0149] After completing the speed parameter optimization at all control points, the differentiated flight path of the UAV is obtained.

[0150] In some embodiments, the smoothness penalty is expressed as:

[0151] The collision penalty is expressed as:

[0152]

[0153] Among them, Nc represents the total number of control points, and Np represents the number of control points corresponding to Q. i The number of {p, v} pairs, i represents the control point index, j represents the index of the {p, v} pair, c ij =S λ(S) -d ij , S λ(S) For each control point Q i The corresponding obstacle avoidance gap, d ij is the control point Q i and the corresponding anchor point P ij The actual distance;

[0154] The dynamic feasibility penalty is expressed as: J dynamic =∑ i＝1 w v F(V i )+∑ i＝1 w a F(Ai )+∑ i＝1 w j F(J i ), where wv, wa, w j is the weight coefficient, and F is the quadratic continuous differentiable function of the higher-order derivative at the i-th control point.

[0155] In some embodiments, the objective function defining the speed parameter optimization is J=λ s J smooth +λ c J collision +λ d J dynamic The steps include:

[0156] Determine whether the anchor semantic label corresponding to the i-th control point is a preset high-cost weight;

[0157] If so, the dynamic feasibility penalty is expressed as:

[0158] Np represents the control point Q i The number of {p, v} pairs, V ij is the direction vector.

[0159] In some specific embodiments, the differentiated flight path planning device further includes:

[0160] an initial image acquisition device, configured to acquire a two-dimensional image and a depth image of the indoor environment, perform semantic segmentation on the two-dimensional image to obtain two-dimensional semantic features, and perform depth estimation on the two-dimensional image to obtain a depth estimation image;

[0161] A completion device, configured to perform depth completion on the depth image according to the depth estimation image to obtain a completed depth image, and extract depth features from the completed depth image;

[0162] A mapping device for mapping the two-dimensional semantic features to a three-dimensional space under the guidance of the depth features, and associating the two-dimensional semantic features with the corresponding depth features to obtain three-dimensional structural features and three-dimensional semantic features;

[0163] A feature fusion device is used to fuse the three-dimensional structural features and the three-dimensional semantic features to obtain a three-dimensional semantic scene graph, wherein each voxel in the three-dimensional semantic scene graph carries three-dimensional semantic information converted from the three-dimensional semantic features corresponding to the voxel.

[0164] The differentiated flight path planning device of this embodiment acquires drone flight parameters and a three-dimensional semantic scene graph of the indoor environment, then combines voxel semantic information with cost weights for path planning. This enables drones to plan more reasonable, safe, and efficient flight paths in complex indoor environments based on their understanding of the semantics of the environment. Compared to traditional path planning methods, this fully considers the semantic attributes of objects in the indoor environment, avoiding high-risk areas and improving the drone's ability to perform missions in complex indoor environments.

[0165] Figure 5 FIG1 shows an internal structure diagram of a computer device in an embodiment. The computer device can be a terminal or a server. Figure 5 As shown, the computer device includes a processor, a memory, and a network interface connected via a system bus. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor may implement a differentiated flight path planning method. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor may implement a differentiated flight path planning method. Those skilled in the art will understand that Figure 5 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present invention and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0166] In one embodiment, a device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the following steps:

[0167] Obtaining flight parameters of a UAV performing a flight mission in an indoor environment, the flight parameters including a flight start point and a flight end point, and obtaining a three-dimensional semantic scene graph of the indoor environment, wherein the three-dimensional semantic scene graph includes a number of voxels;

[0168] Acquire semantic information of each voxel, where the semantic information includes a position vector of the voxel in the indoor environment and a semantic label of the voxel;

[0169] According to a preset semantic label-cost weight mapping relationship, respectively obtaining the cost weight corresponding to the semantic label of each voxel;

[0170] According to the semantic label, cost weight and position vector of each voxel in the three-dimensional semantic scene graph, a flight path from the flight start point to the flight end point is planned to obtain a differentiated flight path of the UAV.

[0171] The differentiated flight path planning device of this embodiment acquires drone flight parameters and a three-dimensional semantic scene graph of the indoor environment, then combines voxel semantic information with cost weights for path planning. This enables drones to plan more reasonable, safe, and efficient flight paths in complex indoor environments based on their understanding of the semantics of the environment. Compared to traditional path planning methods, this fully considers the semantic attributes of objects in the indoor environment, avoiding high-risk areas and improving the drone's ability to perform missions in complex indoor environments.

[0172] In one embodiment, a computer-readable storage medium is provided, storing a computer program. When the computer program is executed by a processor, the processor performs the following steps:

[0173] Obtaining flight parameters of a UAV performing a flight mission in an indoor environment, the flight parameters including a flight start point and a flight end point, and obtaining a three-dimensional semantic scene graph of the indoor environment, wherein the three-dimensional semantic scene graph includes a number of voxels;

[0174] Acquire semantic information of each voxel, where the semantic information includes a position vector of the voxel in the indoor environment and a semantic label of the voxel;

[0175] According to a preset semantic label-cost weight mapping relationship, respectively obtaining the cost weight corresponding to the semantic label of each voxel;

[0176] According to the semantic label, cost weight and position vector of each voxel in the three-dimensional semantic scene graph, a flight path from the flight start point to the flight end point is planned to obtain a differentiated flight path of the UAV.

[0177] The differentiated flight path planning device of this embodiment acquires drone flight parameters and a three-dimensional semantic scene graph of the indoor environment, then combines voxel semantic information with cost weights for path planning. This enables drones to plan more reasonable, safe, and efficient flight paths in complex indoor environments based on their understanding of the semantics of the environment. Compared to traditional path planning methods, this fully considers the semantic attributes of objects in the indoor environment, avoiding high-risk areas and improving the drone's ability to perform missions in complex indoor environments.

[0178] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0179] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0180] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A differentiated flight path planning method, characterized in that: Applied to UAVs, the differentiated flight path planning method includes: Obtaining flight parameters of a UAV performing a flight mission in an indoor environment, the flight parameters including a flight start point and a flight end point, and obtaining a three-dimensional semantic scene graph of the indoor environment, wherein the three-dimensional semantic scene graph includes a number of voxels; Acquire semantic information of each voxel, where the semantic information includes a position vector of the voxel in the indoor environment and a semantic label of the voxel; According to a preset semantic label-cost weight mapping relationship, respectively obtaining the cost weight corresponding to the semantic label of each voxel; According to the semantic label, cost weight and position vector of each voxel in the three-dimensional semantic scene graph, a flight path from the flight start point to the flight end point is planned to obtain a differentiated flight path of the UAV.

2. The differentiated flight path planning method according to claim 1, characterized in that: The cost weights include at least a high cost weight and a low cost weight, wherein the flight obstacle corresponding to the high cost weight is greater than the flight obstacle corresponding to the low cost weight; The step of performing path planning on a flight path from the flight start point to the flight end point based on the semantic label, cost weight, and position vector of each voxel in the three-dimensional semantic scene graph to obtain a differentiated flight path for the UAV includes: performing path planning on a flight path from the flight start point to the flight end point based on a B-spline curve according to the cost weight and position vector of each voxel in the three-dimensional semantic scene graph, so that the flight path preferentially selects voxels passing through low-cost weights, wherein the B-spline curve includes a plurality of control points; For each control point on the B-spline curve, a corresponding obstacle anchor point is assigned on the three-dimensional semantic scene graph, and a semantic label of a voxel where each obstacle anchor point is located is obtained as an anchor point semantic label; According to the anchor point semantic label corresponding to each control point and the cost weight corresponding to the anchor point semantic label, the position of the corresponding control point is adjusted to obtain an adjusted B-spline curve; According to the anchor point semantic label corresponding to each control point, differentiated flight speed parameters are assigned to the adjusted B-spline curve to obtain the differentiated flight path of the UAV.

3. The differentiated flight path planning method according to claim 2, characterized in that: The step of adjusting the position of each control point according to the anchor point semantic label corresponding to each control point and the cost weight corresponding to the anchor point semantic label to obtain the adjusted B-spline curve includes: According to the preset semantic label-cost weight mapping relationship, each control point Q is obtained i Corresponding cost weight λ(S), i=1, 2...N c ; According to the formula S λ(S) =S f-orin ·λ(S), calculate each control point Q separately i The corresponding obstacle avoidance gap S λ(S) , where S f-orin is the preset global safety gap; Adjust each control point Q separately i and the corresponding anchor point P ij The distance from the control point Q i The corresponding obstacle avoidance gap S λ(S) , j is the index of the {p, v} pair, where p represents the anchor point and v represents the repulsive force direction vector associated with the anchor point; The B-spline curve after all control points are adjusted is used as the adjusted B-spline curve.

4. The differentiated flight path planning method according to claim 2, characterized in that: The step of assigning differentiated flight speed parameters to the adjusted B-spline curve according to the anchor point semantic label corresponding to each control point to obtain a differentiated flight path of the UAV includes: The adjusted B-spline curve is parameterized, and each control point in the adjusted B-spline curve is represented as {Q1, Q2, Q3, ..., Q Nc }, set the time interval Δt between adjacent control points to be equal, according to the formula and Calculate the initial velocity parameters of the i-th control point, which include the initial velocity V i , acceleration A i , acceleration J i ; Define the objective function of speed parameter optimization as J = λ s J smooth +λ c J collision +λ d J dynamic , where J smooth For smoothing penalty, J collision For collision penalty, J dynamic is the dynamic feasibility penalty, parameter λ s is the weight of the smoothing penalty, parameter λ c is the weight of the collision penalty, parameter λ d is the weight of dynamic feasibility penalty; For each control point Q i The initial speed parameters are optimized to minimize the objective function Get each control point Q i Optimized speed parameters; After completing the speed parameter optimization at all control points, the differentiated flight path of the UAV is obtained.

5. The differentiated flight path planning method according to claim 4, characterized in that: The smoothing penalty is expressed as: The collision penalty is expressed as: Among them, Nc represents the total number of control points, and Np represents the number of control points corresponding to Q. i The number of {p, v} pairs, i represents the control point index, j represents the index of the {p, v} pair, c ij =S λ(S) -d ij , S λ(S) For each control point Q i The corresponding obstacle avoidance gap, d ij is the control point Q i and the corresponding anchor point P ij The actual distance; The dynamic feasibility penalty is expressed as: J dynamic =∑ i＝1 w v F(V i )+∑ i＝1 w a F(A i )+∑ i＝1 w j F(J i ), where w v , w a , w j is the weight coefficient, and F is the quadratic continuous differentiable function of the higher-order derivative at the i-th control point.

6. The differentiated flight path planning method according to claim 5, characterized in that: The objective function of speed parameter optimization is defined as J=λ s J smooth +λ c J collision +λ d J dynamic The steps include: Determine whether the anchor semantic label corresponding to the i-th control point is a preset high-cost weight; If so, the dynamic feasibility penalty is expressed as: Np represents the control point Q i The number of {p, v} pairs, V ij is the direction vector.

7. The differentiated flight path planning method according to claim 1, characterized in that: Before the step of obtaining the three-dimensional semantic scene graph of the indoor environment, the method includes: Acquire a two-dimensional image and a depth image of the indoor environment, perform semantic segmentation on the two-dimensional image to obtain two-dimensional semantic features, and perform depth estimation on the two-dimensional image to obtain a depth estimation image; Performing depth completion on the depth image according to the depth estimation image to obtain a completed depth image, and extracting depth features in the completed depth image; Under the guidance of the depth feature, the two-dimensional semantic feature is mapped to the three-dimensional space, and the two-dimensional semantic feature is associated with the corresponding depth feature to obtain a three-dimensional structural feature and a three-dimensional semantic feature; The three-dimensional structural features and the three-dimensional semantic features are feature-fused to obtain a three-dimensional semantic scene graph, wherein each voxel in the three-dimensional semantic scene graph carries three-dimensional semantic information converted from the three-dimensional semantic features corresponding to the voxel.

8. A differentiated flight path planning device, characterized in that: Applied to UAVs, the differentiated flight path planning device includes: a data acquisition module, configured to acquire flight parameters of a UAV performing a flight mission in an indoor environment, the flight parameters including a flight start point and a flight end point, and to acquire a three-dimensional semantic scene graph of the indoor environment, wherein the three-dimensional semantic scene graph includes a number of voxels; A semantic acquisition module, configured to acquire semantic information of each voxel, wherein the semantic information includes a position vector of the voxel in the indoor environment and a semantic label of the voxel; A cost weight acquisition module is used to obtain the cost weight corresponding to the semantic label of each voxel according to a preset semantic label-cost weight mapping relationship; The path planning module is used to plan the flight path from the flight starting point to the flight end point according to the semantic label, cost weight and position vector of each voxel in the three-dimensional semantic scene graph, so as to obtain a differentiated flight path of the UAV.

9. A computer-readable storage medium, characterized in that A computer program is stored, and when the computer program is executed by a processor, the processor is caused to perform the steps of the differentiated flight path planning method according to any one of claims 1 to 7.

10. A device, characterized in that The device includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the differentiated flight path planning method according to any one of claims 1 to 7.

Images