A semantic understanding-based unmanned aerial vehicle route planning method and system

Abstract

The application discloses a kind of unmanned aerial vehicle route planning method and system based on semantic understanding, the method steps include: S01. input the voice signal of unmanned aerial vehicle's route planning instruction, and convert voice signal into text data;S02. the text data obtained by conversion is carried out semantic understanding, and the keyword of unmanned aerial vehicle's route planning instruction is extracted;S03. according to current threat area and the keyword of unmanned aerial vehicle's route planning instruction, route planning is carried out, and the planning point coordinate of unmanned aerial vehicle is output;S04. according to the planning point coordinate of unmanned aerial vehicle, the planning trajectory of unmanned aerial vehicle is generated in the simulation environment of unmanned aerial vehicle, and the position information of obstacle, specified target and threat area is mapped to the planning trajectory of unmanned aerial vehicle in simulation environment.The application can realize unmanned aerial vehicle automatic route planning for composite natural language instruction understanding, and has the advantages of simple implementation method, cost, intelligent degree and high efficiency and strong flexibility.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) route planning and control technology, and in particular to a UAV route planning method and system based on semantic understanding. Background Technology

[0002] In practical applications of UAVs, flight path planning is a core function closely related to reconnaissance, strike, and assessment tasks. UAV flight path planning involves setting waypoints to achieve tasks such as automatic flight according to plan and area coverage search for sensor-used areas. In pre-set deterministic scenarios, UAVs can achieve automatic pre-planned flight paths. However, if there are threats such as severe weather or terrain obstacles, UAVs also need to plan a flight path that is as safe as possible and meets their dynamic characteristics.

[0003] Currently, drone flight path planning typically employs a point-to-point approach. This involves planning a path from the starting point to the destination that avoids obstacles, based on a given starting point, ending point, and pre-known obstacle locations in the planning environment. However, this planning method is only suitable for fixed environments and simple planning tasks. It cannot perform dynamic flight path planning for dynamic adversarial environments and complex, ever-changing task requirements. In particular, for complex flight path planning tasks dictated by natural language, human supervision and assistance are often required. Summary of the Invention

[0004] The technical problem to be solved by this invention is: in view of the technical problems existing in the prior art, this invention provides a method and system for unmanned aerial vehicle (UAV) route planning based on semantic understanding that is simple to implement, low in cost, highly intelligent, efficient and flexible.

[0005] To solve the above-mentioned technical problems, the technical solution proposed by this invention is as follows:

[0006] A method for unmanned aerial vehicle (UAV) flight path planning based on semantic understanding, comprising the following steps:

[0007] S01. Input the voice signal of the UAV's flight path planning command and convert the voice signal into text data;

[0008] S02. Perform semantic understanding on the text data obtained in step S01 to extract keywords for the UAV's flight path planning instructions;

[0009] S03. Based on the location information of the current threat area and the keywords of the flight path planning instruction of the UAV obtained in step S02, perform flight path planning and output the coordinates of the UAV's planning point;

[0010] S04. Generate the planned trajectory of the UAV in the UAV simulation environment based on the planned point coordinates of the UAV, and map the position information of obstacles, designated targets and threat areas to the planned trajectory of the UAV in the simulation environment using coordinates.

[0011] Furthermore, the semantic understanding step in step S02 includes:

[0012] The semantic sequence is obtained by treating the text sequence in the input text data as a linear sequence.

[0013] Define a semantic label set: LabelSet = {BA, MA, EA, BO, MO, EO}, where BA represents the current word as the first word of a command, MA represents the current word as the middle word of a command, EA represents the current word as the last word of a command; BO represents the current word as the first word of a region name, MO represents the current word as the middle word of a region name, and EO represents the current word as the last word of a region name.

[0014] An RNN is used to train the semantic sequence to obtain a sequence label in a long sentence;

[0015] According to sequence labeling, command words are composed of BA, MA, and EA, and region words are composed of BO, MO, and EO.

[0016] Furthermore, when annotating semantic sequences, the command name and region name are validated separately. If only the command name or region name exists, the statement is judged to have an error and a reminder message is output.

[0017] Furthermore, after sequence labeling, words are used for semantic understanding using a tree structure. Words with similar meanings in the tree structure are grouped into a set, and then subsets are divided according to the degree of similarity between words in the set.

[0018] Furthermore, in step S03, a segmented UAV flight path planning method based on the fusion of the RRT algorithm and Dubins curves is used to plan the flight path of the UAV. The steps include:

[0019] S301. The route area between the starting point and the ending point is identified and divided into a boundary threat area or a probabilistic threat area. The boundary threat area is the boundary range of the threat area that the UAV needs to avoid, and the probabilistic threat area is the area that the UAV needs to enter to perform the mission.

[0020] S302. Sort the nodes in the boundary threat region and the probabilistic threat region according to the distance between the node and the starting point;

[0021] S303. Sort all nodes in the boundary threat region and the probabilistic threat region according to the shortest path points;

[0022] S304. Determine the type of nodes after the mixed sorting in step S303. If there are consecutive distance points among the nodes, use the Dubins algorithm for path planning. If there are no consecutive distance points, use the RRT (Rapid-exploration Random Tree) algorithm for path planning.

[0023] Furthermore, in step S302, a starting point is first determined, the next node with the shortest distance from the starting point is found, and then the next node is taken as the current node, and the next node with the shortest distance from the current node is found. This process is repeated until the last node is reached, and the sorting value of the shortest path is obtained. The sorting of each node in the corresponding boundary threat area yields the first array Arrt, and the sorting of each node in the corresponding probability threat area yields the second array Arrp.

[0024] Further, in step S303, the distances between the first point Arrp1 in the second array Arrp and each point in the first array Arrt are calculated. If the distances between the first point Arrp1 and the node Arrt are satisfied... j The value of L(Arrt) is the smallest, and L(Arrt) is the smallest. j+1 ) <L(Arrt j-1 Let L(Arrtj+1) represent the distance between Arrtj and Arrt(j+1), and L(Arrtj-1) represent the distance between Arrtj and Arrt(j-1). Then, insert the first point Arrp1 into Arrtj. j With Arrt j+1 Between, otherwise insert into Arrt j With Arrt j-1 Between these, we obtain a new array Arrpt, for the point Arrp in the second array Arrp. i If i > 1, then the new array Arrpt obtained by traversing the array will contain Arrp. i-1 The subsequent values ​​determine the insertion position until the mixed sort of all nodes is complete.

[0025] Furthermore, before step S4, a simulation modeling step of the actual environment of the UAV is also included, including: using Gazebo to build a simulation environment of the UAV, and building a dynamic model of the UAV in the simulation environment, as well as establishing target, threat area and obstacle modeling, and marking the target, obstacle and threat after modeling.

[0026] Furthermore, the target is modeled using a planar circle, the boundary threat area is modeled using a cylinder or cube, and the boundary of the cylinder or cube corresponds to the boundary of the boundary threat area. The probabilistic threat area is modeled using a cone, and the probability of passage in the middle of the cone is lower than the probability of passage at the edge of the cone. Obstacles are modeled using rectangular cubes, and the height of the cube represents the height of the obstacle, and the size of the cube represents the size of the obstacle.

[0027] A UAV route planning system based on a semantic understanding-based UAV route planning method includes:

[0028] The semantic translation module is used to input the voice signal of the drone's flight path planning command and convert the voice signal into text data;

[0029] The semantic understanding module is used to perform semantic understanding on the text data obtained in step S01 and extract keywords for the UAV's flight path planning instructions;

[0030] The flight path planning module is used to plan flight paths based on the location information of the current threat area and the keywords of the flight path planning instructions of the UAV obtained by the semantic understanding module, and output the coordinates of the UAV's planned points.

[0031] The drone simulation module is used to generate the planned trajectory of the drone in the drone simulation environment based on the planned point coordinates of the output drone, and to map the position information of obstacles, designated targets and threat areas into the planned trajectory of the drone through coordinates in the simulation environment.

[0032] A computer system includes a processor and a memory, the memory being used to store a computer program, and the processor being used to execute the computer program to perform the method described above.

[0033] Compared with existing technologies, the advantages of this invention are as follows: This invention combines semantic understanding with flight path planning. By translating the voice signal of flight path planning instructions for UAVs into text, semantic understanding is used to convert the text into flight path planning instructions for UAVs and extract keywords. Based on the keyword information obtained from semantic understanding and the information of threat areas, the corresponding flight path is planned for the UAV instructions. Then, the coordinate point information of the flight path planning is transmitted to the virtual simulation environment of the UAV. The planned path is generated in the simulation environment. At the same time, obstacles, targets, threat areas, etc., established in the simulation environment are mapped to the flight path planning through coordinate mapping. This invention can be applied to complex flight path planning tasks issued by natural language, and efficiently realizes automatic flight path planning for UAVs based on the understanding of complex natural language instructions. Attached Figure Description

[0034] Figure 1This is a schematic diagram illustrating the implementation process of the semantic understanding-based UAV route planning method in this embodiment.

[0035] Figure 2 This is a data interaction diagram of the UAV route planning method based on semantic understanding in this embodiment.

[0036] Figure 3 This is a flowchart illustrating the process of semantic understanding extraction in a specific application embodiment.

[0037] Figure 4 This is a schematic diagram illustrating the implementation process of route planning based on the fusion of the RRT algorithm and Dubins curve in this embodiment.

[0038] Figure 5 This is a schematic diagram illustrating an example of threat source classification.

[0039] Figure 6 This is a schematic diagram illustrating the simulation environment modeling effect of a single target in a specific application embodiment.

[0040] Figure 7 This is a schematic diagram illustrating the simulation environment modeling effect for various types of targets in a specific application embodiment. Detailed Implementation

[0041] The present invention will be further described below with reference to the accompanying drawings and specific preferred embodiments, but this does not limit the scope of protection of the present invention.

[0042] like Figure 1 As shown, the steps of the UAV route planning method based on semantic understanding in this embodiment include:

[0043] S01. Input the voice signal of the UAV's flight path planning command and convert the voice signal into text data;

[0044] S02. Perform semantic understanding on the text data obtained in step S01 to extract keywords for the UAV's flight path planning instructions;

[0045] S03. Based on the location information of the current threat area and the keywords of the UAV flight path planning instruction obtained in step S02, perform flight path planning and output the coordinates of the UAV's planned points;

[0046] S04. Generate the planned trajectory of the UAV in the UAV simulation environment based on the planned point coordinates of the UAV, and map the position information of obstacles, designated targets and threat areas to the planned trajectory of the UAV in the simulation environment through coordinates.

[0047] In manned-unmanned collaborative missions, the use of natural language interaction to enable manned pilots to command and control unmanned aerial vehicles (UAVs) can achieve an efficient control mode. This embodiment combines semantic understanding with flight path planning, and through speech-to-text conversion, semantic understanding, flight path planning, and UAV environment simulation, it realizes flight path planning oriented towards semantic understanding. This makes it applicable to complex flight path planning tasks issued by natural language, and efficiently realizes automatic flight path planning for UAVs oriented towards understanding complex natural language commands.

[0048] like Figure 2 As shown, the process first involves translating the voice signal of the flight path planning command for the UAV into text. Through semantic understanding, the text is converted back into the UAV's flight path planning command, and keywords are extracted. Based on the keyword information obtained from semantic understanding and information about threat areas, the corresponding flight path is planned for the UAV, and the coordinates of the UAV's movement, i.e., waypoint coordinates, are output. Then, a simulation environment is launched, and the coordinate information of the flight path planning is transmitted to the UAV's virtual simulation environment. The planned path is generated in this simulation environment, and obstacles, targets, and threat areas established in the simulation environment are mapped to the flight path planning using coordinates. When it is necessary to readjust the planning task or execute a new planning task, the process returns to step S01 and the voice signal of the UAV's flight path planning command is re-entered to complete the adjustment of the UAV's flight path planning, thus realizing UAV flight path planning based on semantic understanding.

[0049] In step S01 of this embodiment, the translation of the speech signal into text can be achieved using a speech conversion tool, such as iFlytek or Baidu Voice, and the online speech-to-text function of the speech conversion tool can be used for online translation.

[0050] The speech signal is converted into text commands, and then semantic understanding is performed to derive the commands and parameters from the text commands. Semantic understanding involves extracting and recognizing keywords in the UAV commands from the text data. For example, if the original text data is "bypass obstacle 1", after text processing, the command "bypass" and the parameter content of "obstacle 1" are obtained. The extracted command and parameter content are then passed to the subsequent flight path planning algorithm for flight path planning.

[0051] In natural language processing, sequence labeling is crucial. Semantic tag slot acquisition is similar to sequence labeling, involving labeling each element in a linear sequence. This is equivalent to sequence labeling the content of a sentence, which involves segmenting the text according to context and a specific order. The key is to provide a suitable label for the content based on the linear labeling requirements. This example uses semantic tags (command names and region names) to illustrate how an RNN model achieves semantic extraction. First, the input text sequence is considered as a linear sequence of individual characters (specifically Chinese characters) as the object and basis for semantic extraction. The object of semantic extraction is this linear sequence of individual characters.

[0052] The semantic understanding steps in step S02 of this embodiment include:

[0053] The semantic sequence is obtained by treating the text sequence in the input text data as a linear sequence.

[0054] Define a semantic label set: LabelSet = {BA, MA, EA, BO, MO, EO}, where BA represents the current word as the first word of a command, MA represents the current word as the middle word of a command, EA represents the current word as the last word of a command; BO represents the current word as the first word of a region name, MO represents the current word as the middle word of a region name, and EO represents the current word as the last word of a region name.

[0055] The semantic sequence is trained using an RNN (Recurrent Neural Network) to obtain a sequence label in a long sentence;

[0056] According to sequence labeling, command words are composed of BA, MA, and EA, and address words are composed of BO, MO, and EO.

[0057] That is, define a semantic tag set. After obtaining the input word sequence and the semantic tag set, according to the sequence label, BA, MA and EA form the command word, and BO, MO and EO form the address word.

[0058] In this embodiment, the language collected is Chinese. Semantic understanding of the collected Chinese is performed to convert it into flight path planning instructions for the UAV. Due to the complexity of Chinese semantics, it is difficult to extract the semantic meaning from a single word or phrase (e.g., "change course," "course has changed"), and errors may occur during speech-to-text conversion. Therefore, this embodiment uses an RNN (Recurrent Neural Network) for training to obtain a sequence label within a long sentence. The input to the RNN is a long sentence, and the output is a sequence label within that sentence, such as... Figure 3The image shows an example of semantic understanding extraction using the method described above in a specific application embodiment ("Bypassing region 1, covering and searching region 2, and attacking target 3"). For the RNN engine, the input is "Bypassing region 1, covering and searching region 2, and attacking target 3", and the output is ( / BA / EA / BO / MO / MO / EO / BA / MA / MA / EA / BO / MO / EO / BA / EA / BO / MO / MO / EO). Based on the sequence labeling, BA, MA, and EA form the Command word, and BO, MO, and EO form the Address word. A Command+Address phrase is then established, starting with a pause in the speech.

[0059] In this embodiment, semantic sequence annotation is performed by validating both command names and region names. If only a command name or region name exists, the statement is deemed to have an error, and a warning message is output. That is, semantic sequence annotation validation uses a Command+Address approach. If only Command or Address exists, the corresponding short statement in the task is deemed to have an error, and a warning message is issued to alert the operator.

[0060] In this embodiment, words after sequence labeling are semantically understood using a tree structure. Words with similar meanings in the tree structure are grouped into a set, and further subdivided into subsets based on the degree of similarity between words within the set. For example, "reach," "arrive," "pass through," and "strike" all have the purpose of "reaching" a certain area target, represented by set ①; "bypass" and "around" both have the purpose of "bypassing" a certain area target, represented by set ②; "monitor" and "track" both have the purpose of "tracking," represented by set ③; "target number 1," "area number 1," and "target number 2," etc., all have the purpose of "area," represented by set ④. Set ① is further subdivided into subsets, with "reach" and "arrive," which have the closest meanings, forming subset ①, "pass through" forming subset ②, and "strike" forming subset ③. Then, using the tree structure, taking the input "strike" as an example, it needs to pass through sets ①-③ to select the appropriate planning method. The planning method used is "accelerate" to pass through the target and "strike" the target.

[0061] After obtaining the drone flight path planning instructions through semantic understanding, it is further necessary to plan the drone flight path based on these instructions. For example... Figure 4 As shown, in step S03 of this embodiment, a segmented UAV flight path planning method based on the fusion of the RRT (Rapidly-exploring Random Tree) algorithm and the Dubins curve is specifically used to plan the flight path of the UAV. The steps include:

[0062] S301. The route area between the starting point and the ending point is identified and divided into a boundary threat area or a probabilistic threat area. The boundary threat area is the boundary range of the threat area that the UAV needs to avoid, and the probabilistic threat area is the area that the UAV needs to enter to perform the mission.

[0063] S302. Sort the nodes in the boundary threat area and the probabilistic threat area according to the distance between the node and the starting point;

[0064] S303. Sort all nodes in the boundary threat region and the probabilistic threat region according to the shortest path points;

[0065] S304. Determine the type of nodes after the mixed sorting in step S303. If there are consecutive distance points among the nodes, use the Dubins algorithm for path planning; otherwise, use the RRT algorithm for path planning.

[0066] The Randomized Tracing Tree (RRT) algorithm is a sampling-based path planning method with high search efficiency in multi-dimensional space. Initially, the RRT algorithm uses the starting point of the random tree as its root node, and continuously generates new nodes through random sampling. These nodes connect to form an outward-expanding random tree. The search is successful when the target point is also included in the random tree. The Dubins curve algorithm, on the other hand, solves the problem of finding the shortest path for an object given its starting and ending positions and known directions of movement. While efficient, the Dubins curve algorithm requires known starting and ending positions, and the line connecting them must be free of obstacles; that is, the threat regions at the starting and ending points must be of the same type. RRT, as an efficient search method, allows the starting and ending points to include other points or different threat regions. This embodiment combines the Dubins and RRT algorithms, leveraging the advantages of both to achieve efficient and accurate UAV path planning.

[0067] This embodiment categorizes threats encountered by the UAV during mission execution into two types: boundary threat areas and probabilistic threat areas. The boundary threat area refers to the boundary range of the threat area, i.e., the area the UAV needs to avoid. Figure 5 Obstacles and attack zones are areas where threats have a certain probability of causing damage or even destruction to a drone once it enters. Therefore, drones should avoid these areas during path planning. Probability threat zones are threat areas defined relative to boundary threat zones, such as... Figure 5The thunderstorm areas and radar detection areas shown are also threats to drones. However, since drones need to complete preset tasks or need to cross these areas, and there is no absolute threat to the drones' status, for example, when planning the path, it is necessary to consider how to cover and search these areas and plan the attack route for the target.

[0068] In step S302 of this embodiment, a starting point is first determined, and the next node with the shortest distance to the starting point is found. Then, the next node is used as the current node, and the next node with the shortest distance to the current node is found. This process is repeated until the last node is reached, resulting in the sorted values ​​of the shortest paths. The sorted values ​​of nodes in the boundary threat area are used to obtain the first array Arrt, and the sorted values ​​of nodes in the probabilistic threat area are used to obtain the second array Arrp. For example, a starting point A is determined, the distance between starting point A and other points is calculated, and the point B with the shortest distance is found. Then, point B is compared with other points (except point A) to find the point C with the shortest distance. This process is repeated until the last point is reached, thus obtaining the sorted values ​​of the shortest paths.

[0069] In step S303 of this embodiment, the distance between the first point Arrp1 in the second array Arrp and each point in the first array Arrt is calculated. If the distance between the first point Arrp1 and the node Arrt is satisfied... j The value of L(Arrt) is the smallest, and L(Arrt) is the smallest. j+1 ) <L(Arrt j-1 Let L(Arrtj+1) represent the distance between Arrtj and Arrt(j+1), and L(Arrtj-1) represent the distance between Arrtj and Arrt(j-1). Then, insert the first point Arrp1 into Arrtj. j With Arrt j+1 Between, otherwise insert into Arrt j With Arrt j-1 Between these, we obtain a new array Arrpt, for the point Arrp in the second array Arrp. i If i > 1, then the new array Arrpt obtained by traversing the array will contain Arrp. i-1 The subsequent values ​​determine the insertion position. That is, when i > 1, Arrpi only needs to traverse the values ​​after Arrpi-1 in the newly inserted array Arrpt until the mixed sort of all nodes is completed. The distance calculated above can specifically be the Euclidean distance.

[0070] like Figure 4 As shown, N is obtained by sorting the nodes in the boundary threat region according to their distance from the starting point. i1 N i2 N i3 ... NiN N is the number of nodes. M is obtained by sorting the nodes in the probabilistic threat region according to their distance from the starting point. i1 M i2 M i3 M iN After sorting by shortest path points, N is formed. i1 M i1 N i2 M i2 ... N iN M iN Determine whether there exists a continuous N iN M iN If so, the Dubins algorithm is used for path planning; otherwise, the RRT algorithm is used.

[0071] Before step S4, the above steps also include a simulation modeling step for the actual environment of the UAV, to construct a three-dimensional simulation task environment for various threats and tasks. The simulation modeling specifically includes: building a simulation environment for the UAV using the Gazebo simulation platform, and building a dynamic model of the UAV within the simulation environment; that is, the Gazebo simulation environment contains a dynamic model of the UAV; and establishing models for targets, threat areas, and obstacles, and then marking these targets, obstacles, and threats. For example, when marking targets and threats, labels such as "Target 1," "Target 2," "Threat 1," and "Threat 2" are used to reduce the problem of low recognition rates due to the large number of numbers in speech-to-text conversion. After flight path planning based on semantic understanding, the results of the flight path planning are transmitted to the Gazebo virtual simulation environment in the form of coordinates. In the virtual simulation environment, this is presented as the UAV moving in three-dimensional space, to intuitively show the UAV's state in the environment, facilitating operators to adjust flight path planning commands based on the UAV's real-time status.

[0072] In this embodiment, the target is modeled as a planar circle, the boundary threat region is modeled as a cylinder or cube, and the boundary of the cylinder or cube corresponds to the boundary of the boundary threat region. The probabilistic threat region is modeled as a cone, and the probability of passage in the middle of the cone is lower than that at the edge of the cone. In the ROS simulation environment, obstacles are modeled using rectangular cubes, where the height of the cube represents the height of the obstacle, and the size of the cube represents the size of the obstacle. Additionally, severe weather is modeled using cylinders; reconnaissance targets are modeled as circles, with the cylinder height set to zero, meaning they do not pose a threat to the airspace. After the model is built, obstacles, threats, and other targets are labeled, such as... Figure 6 The image shows a simulation environment model for a single target, such as 1 representing an obstacle, 2 representing severe weather, and 3 representing a reconnaissance target. Figure 7 The image shows a simulation environment model for various types of targets, where 1 represents obstacles, 2 represents weather, 3 represents targets, 4 represents threat targets, and 5 represents weather.

[0073] like Figure 2 As shown, firstly, a model and representation of the threat region are established in Gazebo, and the location information of the threat region is mapped onto the flight path planning map. By recording the location of the threat region in the simulation environment and sending it to the flight path planning module, the environment for flight path planning is generated. The input to flight path planning is information about the threat region and keywords. For example, the keyword information is "the UAV bypasses area 1 and flies to area 2". The output of flight path planning is the coordinate information of the points on the planned path. The coordinate information of the planned path points is sent to the UAV to control the UAV's flight, so as to meet the requirements of flight path planning based on task semantic understanding and prepare for subsequent UAV missions. Then, the starting position and ending position of the UAV are sent to the simulation environment to generate the starting and ending positions in the flight path planning.

[0074] This embodiment of the UAV route planning system based on semantic understanding includes:

[0075] The semantic translation module is used to input the voice signal of the drone's flight path planning command and convert the voice signal into text data;

[0076] The semantic understanding module is used to perform semantic understanding on the text data converted by the semantic translation module, and convert it into flight path planning instructions for the UAV.

[0077] The flight path planning module is used to plan the flight path according to the flight path planning instructions of the UAV obtained by the semantic understanding module, and output the coordinate points of the UAV during its movement.

[0078] The drone simulation module is used to generate the planned trajectory of the drone in a simulation environment based on the coordinate points of the drone during its movement, and to map obstacles and specified targets to the planned trajectory of the drone in the simulation environment using coordinates.

[0079] The simulation environment of the aforementioned UAV simulation module is set up independently from the semantic understanding module and the flight path planning module. Data interaction between the simulation environment and the semantic understanding module and the flight path planning module is achieved through network transmission, thereby enabling multi-UAV control, manned / UAV collaboration, etc.

[0080] The UAV route planning system based on the semantic understanding method in this embodiment corresponds one-to-one with the UAV route planning system based on the semantic understanding method described above, and will not be described again here.

[0081] This embodiment also provides a computer system, including a processor and a memory, wherein the memory is used to store a computer program and the processor is used to execute the computer program to perform the method described above.

[0082] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the invention. Therefore, any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention should fall within the protection scope of the present invention.

Claims

1. A method for unmanned aerial vehicle (UAV) route planning based on semantic understanding, characterized by the following steps: include: S01. Input the voice signal of the UAV's flight path planning command and convert the voice signal into text data; S02. Perform semantic understanding on the text data obtained in step S01 to extract keywords for the UAV's flight path planning instructions; S03. Based on the location information of the current threat area and the keywords of the flight path planning instruction of the UAV obtained in step S02, perform flight path planning and output the coordinates of the UAV's planning point; S04. Generate the planned trajectory of the UAV in the UAV simulation environment based on the planned point coordinates of the UAV, and map the position information of obstacles, designated targets and threat areas to the planned trajectory of the UAV in the simulation environment through coordinates; The semantic understanding steps in step S02 include: The semantic sequence is obtained by treating the text sequence in the input text data as a linear sequence. Define a semantic label set: LabelSet = {BA,MA,EA,BO,MO,EO}, where BA represents the current word as the first word of a command, MA represents the current word as the middle word of a command, EA represents the current word as the last word of a command; BO represents the current word as the first word of a region name, MO represents the current word as the middle word of a region name, and EO represents the current word as the last word of a region name. An RNN is used to train the semantic sequence to obtain a sequence label in a long sentence; According to sequence labeling, command words are composed of BA, MA, and EA, and region words are composed of BO, MO, and EO; In step S03, a segmented UAV flight path planning method based on the fusion of the RRT algorithm and Dubins curve is used to plan the flight path of the UAV. The steps include: S301. The route area between the starting point and the ending point is identified and divided into a boundary threat area or a probabilistic threat area. The boundary threat area is the boundary range of the threat area that the UAV needs to avoid, and the probabilistic threat area is the area that the UAV needs to enter to perform the mission. S302. Sort the nodes in the boundary threat region and the probabilistic threat region according to the distance between the node and the starting point; S303. Sort all nodes in the boundary threat region and the probabilistic threat region according to the shortest path points; S304. Determine the type of nodes after the mixed sorting in step S303. If there are consecutive distance points among the nodes, use the Dubins algorithm for path planning; otherwise, use the RRT algorithm for path planning.

2. The UAV route planning method based on semantic understanding according to claim 1, characterized in that, When annotating semantic sequences, the use of command names and region names is validated separately. If only the command name or region name exists, the statement is judged to have an error and a reminder message is output. After sequence labeling, words are used for semantic understanding using a tree structure. Words with similar meanings in the tree structure are grouped into a set, and then subsets are formed based on the degree of similarity between words in the set.

3. The UAV route planning method based on semantic understanding according to claim 1, characterized in that, In step S302, a starting point is first determined, the next node with the shortest distance from the starting point is found, and then the next node with the shortest distance from the current node is found. This process is repeated until the last node is reached, and the sorting value of the shortest path is obtained. The sorting of each node in the corresponding boundary threat area yields the first array Arrt, and the sorting of each node in the corresponding probability threat area yields the second array Arrp.

4. The UAV route planning method based on semantic understanding according to claim 3, characterized in that, In step S303, the distance between the first point Arrp1 in the second array Arrp and each point in the first array Arrt is calculated. If the distance between the first point Arrp1 and the node Arrt is satisfied... j The value of L(Arrt) is the smallest, and L(Arrt) is the smallest. j+1 ) <L(Arrt j-1 Let L(Arrtj+1) represent the distance between Arrtj and Arrt(j+1), and L(Arrtj-1) represent the distance between Arrtj and Arrt(j-1). Then, insert the first point Arrp1 into Arrtj. j With Arrt j+1 Between, otherwise insert into Arrt j With Arrt j-1 Between these, a new array Arrpt is obtained, for the point Arrp in the second array Arrp. i If i > 1, then the new array Arrpt obtained by traversing the array will contain Arrp. i-1 The subsequent values ​​determine the insertion position until the mixed sort of all nodes is complete.

5. The UAV route planning method based on semantic understanding according to any one of claims 1 to 4, characterized in that, Before step S4, there is also a simulation modeling step for the actual environment of the UAV, including: using Gazebo to build a simulation environment for the UAV, and building a dynamic model of the UAV in the simulation environment, as well as establishing models of targets, threat areas and obstacles, and marking targets, obstacles and threats after modeling.

6. The UAV route planning method based on semantic understanding according to claim 5, characterized in that, The target is modeled as a planar circle, the boundary threat area is modeled as a cylinder or cube, and the boundary of the cylinder or cube corresponds to the boundary of the boundary threat area. The probabilistic threat area is modeled as a cone, and the probability of passage in the middle of the cone is lower than the probability of passage at the edge of the cone. Obstacles are modeled as rectangular cubes, and the height of the cube represents the height of the obstacle, and the size of the cube represents the size of the obstacle.

7. A system for implementing the UAV route planning method based on semantic understanding as described in any one of claims 1 to 6, characterized in that, include: The semantic translation module is used to input the voice signal of the drone's flight path planning command and convert the voice signal into text data; The semantic understanding module is used to perform semantic understanding on the text data obtained in step S01 and extract keywords for the UAV's flight path planning instructions; The flight path planning module is used to plan flight paths based on the location information of the current threat area and the keywords of the flight path planning instructions of the UAV obtained by the semantic understanding module, and output the coordinates of the UAV's planned points. The drone simulation module is used to generate the planned trajectory of the drone in the drone simulation environment based on the planned point coordinates of the output drone, and to map the position information of obstacles, designated targets and threat areas into the planned trajectory of the drone through coordinates in the simulation environment.

8. A computer system comprising a processor and a memory, the memory being used to store computer programs, characterized in that, The processor is used to execute the computer program to perform the method as described in any one of claims 1 to 6.

Images