A low-altitude air navigation support method and system based on artificial intelligence

Abstract

The present invention relates to the field of air navigation support technology, and discloses an artificial intelligence-based low-altitude air navigation support method and system. The artificial intelligence-based low-altitude air navigation support method includes the following steps: obtaining aircraft position information, aircraft flight status information, aircraft surround view images, destination location information, surrounding terrain data, airspace control information, historical flight data, and historical navigation instruction data; deriving a virtual navigation beacon position set based on the obtained information; obtaining an augmented reality navigation scene image based on the obtained information and the virtual navigation beacon position set; obtaining natural language navigation instructions based on the obtained information and the virtual navigation beacon position set; and integrating the augmented reality navigation scene image and the natural language navigation instructions into a flight control interface. The present invention considers the complexity of the airspace, combines a set priority function with virtual navigation beacons, generates an augmented reality navigation scene image, and thereby makes navigation information more intuitive and accurate.

Description

Technical Field

[0001] The present invention relates to the field of aviation support technology, and more specifically, to a low-altitude aviation support method and system based on artificial intelligence. Background Art

[0002] Low-altitude aircraft require highly accurate navigation to navigate complex terrain, urban airspace, and other specific scenarios. Traditional navigation methods are not intuitive enough for pilots, making it difficult to quickly and accurately obtain navigation information. Therefore, a new navigation system combining virtual navigation aids and augmented reality technology is needed to optimize air traffic control.

[0003] In high-density airspace scenarios, such as those around busy airports and in controlled airspace areas for large events, there are numerous aircraft, and airspace control information is complex and frequently changing. This can easily cause pilots to become confused by the overwhelming information, compromising navigation accuracy and timeliness. Therefore, a navigation method that can be applied to complex, high-density scenarios is urgently needed to ensure low-altitude flight for low-altitude aircraft. Summary of the Invention

[0004] The present invention provides an artificial intelligence-based low-altitude navigation support method and system to solve the technical problems in related technologies in high-density airspace scenarios, where there are a large number of aircraft and airspace control information is complex and changes frequently, affecting the accuracy and timeliness of navigation.

[0005] The present invention provides a low-altitude air navigation support method based on artificial intelligence, comprising:

[0006] Step 100: Acquire aircraft position information, aircraft flight status information, aircraft surround view images, destination location information, surrounding terrain data, airspace control information, historical flight data, and historical navigation instruction data;

[0007] Flight status information includes: flight speed, flight direction;

[0008] The flight direction includes the aircraft's yaw, pitch, and roll angles;

[0009] Step 200: Based on the acquired information, a virtual navigation mark is set by using a deep reinforcement learning model to obtain a virtual navigation mark position set;

[0010] Step 300: Based on the acquired information and the virtual navigation mark position set obtained by the deep reinforcement learning model, a deep convolutional neural network model is used to perform real-time rendering of the augmented reality scene to obtain an augmented reality navigation scene image;

[0011] Step 400: Using the acquired information and the virtual navigation mark position set derived by the deep reinforcement learning model as input to the Seq2Seq model, outputting natural language navigation instructions;

[0012] Step 500: Integrate augmented reality navigation scene images and natural language navigation instructions into the flight control interface to obtain pilot operation feedback.

[0013] In a preferred embodiment, the aircraft surround view images are collected in real time by a multi-angle camera array and synchronized with the inertial measurement unit data to calibrate the image space coordinates.

[0014] In a preferred embodiment, in step 200, the virtual navigation mark setting problem is modeled as a Markov decision process, and the specific steps include:

[0015] Step 201: Acquire destination location information, surrounding terrain data, airspace control information, and historical flight data;

[0016] Step 202: Build a deep reinforcement learning model, define the state space S, the action space A, and design the reward function R;

[0017] Step 203, setting a priority function for virtual navigation mark settings based on the airspace busyness and the proximity of the aircraft to other targets;

[0018] Step 204 , calculating a priority value for each virtual navigation mark position in the historical flight data and integrating the obtained priority of each virtual navigation mark position into the state S;

[0019] Step 205: Initialize the deep reinforcement learning model and train the deep reinforcement learning model using historical flight data;

[0020] Step 206: Generate a virtual navigation mark position set using the learned strategy based on the current state data.

[0021] In a preferred embodiment, the state space S includes: aircraft position, aircraft flight status information, surrounding terrain data, and airspace control information;

[0022] The action space A includes: combinations of various possible positions of the virtual navigation mark in the airspace;

[0023] Each possible position in the airspace is discretized by constructing a three-dimensional spatial grid model to divide the airspace into a set of discrete coordinate points with a preset accuracy, and a digital storage system for the set of coordinate points is constructed based on a spatial database.

[0024] In a preferred embodiment, the reward function R is as follows:

[0025] ;

[0026] : Safety Reward;

[0027] : efficiency reward;

[0028] : Energy consumption penalty;

[0029] : Regulatory compliance rewards;

[0030] 、 、 、 : The first, second, third and fourth multi-objective weight coefficients;

[0031] ;

[0032] : The minimum real-time distance between the aircraft and obstacles;

[0033] : safety distance threshold;

[0034] : distance attenuation coefficient, the default value is 50;

[0035] ;

[0036] : The deviation distance between the aircraft's current position and the planned route;

[0037] : the total length of the planned route;

[0038] ;

[0039] : The number of virtual navigation marks currently set;

[0040] : The maneuvering frequency of the aircraft per unit time;

[0041] 、 : The first and second energy consumption weight coefficients, the default values ​​are 0.1 and 0.5 respectively;

[0042] ;

[0043] : Number of violations of control rules;

[0044] : Penalty coefficient of regulatory rules.

[0045] In a preferred embodiment, the priority function is as follows:

[0046] ;

[0047] ;

[0048] ;

[0049] ;

[0050] : airspace busy coefficient;

[0051] : The real-time distance between the aircraft and the target point, i.e. the virtual beacon;

[0052] : The first dynamic weight coefficient;

[0053] : The second dynamic weight coefficient;

[0054] : priority function;

[0055] : The number of aircraft in the current airspace;

[0056] : Maximum aircraft capacity of the airspace.

[0057] In a preferred embodiment, the specific steps of step 300 include:

[0058] Step 301: Acquire aircraft position information, aircraft flight status information, virtual navigation mark position set, aircraft surround view image, surrounding terrain data, and airspace control information;

[0059] Step 302: construct a deep convolutional neural network model;

[0060] Step 303: Set the update frequency function of the airspace control information and update the airspace control information;

[0061] In step 304, the acquired information is input into a convolutional neural network model, and after a series of feature extraction and information fusion operations, an augmented reality navigation scene image is generated.

[0062] In a preferred embodiment, the specific steps of step 400 include:

[0063] Step 401, obtaining aircraft position information, aircraft flight status information, virtual navigation mark position set, and historical navigation instruction data;

[0064] Step 402: Build a Seq2Seq model.

[0065] Step 403: Use historical data to train and learn the mapping relationship between state feature sequences and natural language navigation instructions;

[0066] Step 404: Set an update frequency function for airspace control information and update the airspace control information.

[0067] Step 405: Encode the acquired data into a feature vector sequence and input it into a Seq2Seq model, output and decode it to generate natural language navigation instructions.

[0068] In a preferred embodiment, the pilot operation feedback in step 500 includes: instruction confirmation and instruction correction. If the feedback is instruction correction, steps 100-500 are immediately re-executed.

[0069] An artificial intelligence-based low-altitude navigation support system is used to execute any of the artificial intelligence-based low-altitude navigation support methods described above.

[0070] The beneficial effect of the present invention is that the present invention generates an augmented reality navigation scene image by considering the complexity of the airspace, combining a set priority function and a virtual navigation mark, thereby making the navigation information more intuitive and accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 This is a flow chart of a low-altitude air navigation support method based on artificial intelligence of the present invention;

[0072] Figure 2 This is an example of information acquisition of a low-altitude air navigation support method based on artificial intelligence of the present invention;

[0073] Figure 3 This is an operational example of the artificial intelligence-based low-altitude navigation support system of the present invention. DETAILED DESCRIPTION

[0074] The subject matter described herein will now be discussed with reference to example embodiments. It should be understood that these embodiments are discussed solely to enable those skilled in the art to better understand and implement the subject matter described herein, and that the functions and arrangements of the elements discussed may be varied without departing from the scope of this specification. Various examples may omit, substitute, or add various processes or components as needed. Furthermore, features described in some examples may be combined in other examples.

[0075] At least one embodiment of the present invention discloses a low-altitude air navigation support method based on artificial intelligence, such as Figure 1 As shown, the following steps are included:

[0076] Step 100: Acquire aircraft position information, aircraft flight status information, aircraft surround view images, destination location information, surrounding terrain data, airspace control information, historical flight data, and historical navigation instruction data;

[0077] Flight status information includes: flight speed, flight direction;

[0078] The flight direction includes the aircraft's yaw angle, pitch angle, and roll angle.

[0079] In one embodiment of the present invention, the aircraft surround view images need to be collected in real time by a multi-angle camera array and synchronized with inertial measurement unit (IMU) data to calibrate the image space coordinates.

[0080] Step 200: Based on the acquired information, a virtual navigation mark is set by using a deep reinforcement learning model to obtain a virtual navigation mark position set;

[0081] In one embodiment of the present invention, in step 200, the virtual navigation mark setting problem is modeled as a Markov decision process, and the specific steps include:

[0082] Step 201: Acquire destination location information, surrounding terrain data, airspace control information, and historical flight data;

[0083] Step 202: Build a deep reinforcement learning model, define the state space S, the action space A, and design the reward function R;

[0084] In some embodiments of the present invention, the state space S includes: aircraft position, aircraft flight state information, surrounding terrain data, and airspace control information;

[0085] The action space A includes: combinations of various possible positions of the virtual navigation mark in the airspace;

[0086] It should be noted that each possible position in the airspace is discretized by constructing a three-dimensional spatial grid model to divide the airspace into a set of discrete coordinate points with a preset accuracy, and a digital storage system of the coordinate point set is constructed based on a spatial database.

[0087] The reward function R is as follows:

[0088] ;

[0089] : Safety Reward;

[0090] : efficiency reward;

[0091] : Energy consumption penalty;

[0092] : Regulatory compliance rewards;

[0093] 、 、 、 : The first, second, third and fourth multi-objective weight coefficients;

[0094] ;

[0095] : The minimum real-time distance between the aircraft and obstacles;

[0096] : safety distance threshold;

[0097] : distance attenuation coefficient, the default value is 50;

[0098] ;

[0099] : The deviation distance between the aircraft's current position and the planned route;

[0100] : the total length of the planned route;

[0101] ;

[0102] : The number of virtual navigation marks currently set;

[0103] : The maneuvering frequency of the aircraft per unit time;

[0104] 、 : The first and second energy consumption weight coefficients, the default values ​​are 0.1 and 0.5 respectively;

[0105] ;

[0106] : Number of violations of control rules;

[0107] : Penalty coefficient of regulatory rules;

[0108] Step 203, setting a priority function for virtual navigation mark settings based on the airspace busyness and the proximity of the aircraft to other targets;

[0109] In one embodiment of the present invention, the priority function is as follows:

[0110] ;

[0111] ;

[0112] ;

[0113] ;

[0114] : airspace busy coefficient;

[0115] : The real-time distance between the aircraft and the target point, i.e. the virtual beacon;

[0116] : The first dynamic weight coefficient;

[0117] : The second dynamic weight coefficient;

[0118] : priority function;

[0119] : The number of aircraft in the current airspace;

[0120] : Maximum aircraft capacity of the airspace;

[0121] Step 204 , calculating a priority value for each virtual navigation mark position in the historical flight data and integrating the obtained priority of each virtual navigation mark position into the state S;

[0122] Step 205: Initialize the deep reinforcement learning model and train the deep reinforcement learning model using historical flight data;

[0123] Step 206: Generate a virtual navigation mark position set using the learned strategy based on the current state data.

[0124] In one embodiment of the present invention, when the deep reinforcement learning model outputs an empty navigation mark set, indicating that the current airspace is extremely complex, it switches to the preset emergency route and triggers manual takeover.

[0125] Step 300: Based on the acquired information and the virtual navigation mark position set obtained by the deep reinforcement learning model, a deep convolutional neural network model is used to perform real-time rendering of the augmented reality scene to obtain an augmented reality navigation scene image;

[0126] In one embodiment of the present invention, the specific steps of step 300 include:

[0127] Step 301: Acquire aircraft position information, aircraft flight status information, virtual navigation mark position set, aircraft surround view image, surrounding terrain data, and airspace control information;

[0128] Step 302: construct a deep convolutional neural network model;

[0129] Step 303: Set the update frequency function of the airspace control information and update the airspace control information;

[0130] In step 304, the acquired information is input into a convolutional neural network model, and after a series of feature extraction and information fusion operations, an augmented reality navigation scene image is generated.

[0131] Step 400 : Using the acquired information and the virtual navigation mark position set derived by the deep reinforcement learning model as input to a Seq2Seq (Sequence to Sequence) model, outputting natural language navigation instructions;

[0132] In one embodiment of the present invention, the specific steps of step 400 include:

[0133] Step 401, obtaining aircraft position information, aircraft flight status information, virtual navigation mark position set, and historical navigation instruction data;

[0134] Step 402: Build a Seq2Seq model.

[0135] Step 403: Use historical data to train and learn the mapping relationship between state feature sequences and natural language navigation instructions;

[0136] Step 404: Set an update frequency function for airspace control information and update the airspace control information.

[0137] Step 405: Encode the acquired data into a feature vector sequence and input it into a Seq2Seq model, output and decode it to generate natural language navigation instructions.

[0138] In one embodiment of the present invention, the update frequency of the airspace control information shares the same update frequency function as step 300, and the airspace information of the two modules is synchronized through the central data bus.

[0139] Step 500: Integrate augmented reality navigation scene images and natural language navigation instructions into the flight control interface to obtain pilot operation feedback.

[0140] In one embodiment of the present invention, the augmented reality navigation scene image is projected using an optical perspective head display, and the virtual navigation mark and the real scene are aligned in real time using SLAM (Simultaneous Localization and Mapping) technology.

[0141] In one embodiment of the present invention, natural language is displayed through speech synthesis and text scrolling, and high-priority instructions are highlighted in red as a warning.

[0142] In one embodiment of the present invention, a conflict resolution mechanism is added to trigger the deep reinforcement learning model in step 200 to regenerate navigation marks when a logical conflict occurs between AR navigation and voice commands.

[0143] In one embodiment of the present invention, the pilot operation feedback in step 500 includes: command confirmation and command correction. If the feedback is a command correction, steps 100-500 are immediately re-executed.

[0144] In one embodiment of the present invention, the inference time constraint of the above-mentioned deep reinforcement learning model, deep convolutional neural network model and Seq2Seq model is less than a fixed duration, and the default value is 100ms.

[0145] In one embodiment of the present invention, the loss function when the deep convolutional neural network model is trained is as follows:

[0146] ;

[0147] ;

[0148] ;

[0149] ;

[0150] ;

[0151] ;

[0152] ;

[0153] : represents the structural similarity index loss function;

[0154] : Minimize function;

[0155] : Represents the real augmented reality navigation scene image;

[0156] : Augmented reality navigation scene images predicted by the model;

[0157] : is a real image The average pixel value of

[0158] : Predicted image The average pixel value of

[0159] : the total number of pixels in the image;

[0160] :image Middle The pixel value of each pixel;

[0161] : Predicted image Middle The pixel value of each pixel;

[0162] : Real image The standard deviation of pixel values;

[0163] : Predicted image The standard deviation of pixel values;

[0164] : Real image and predicted images The covariance between

[0165] 、 : To avoid the denominator being 0, set the first and second constants to be smaller;

[0166] In one embodiment of the present invention, the loss function when the Seq2Seq model is trained is as follows:

[0167] ;

[0168] : weighted cross entropy loss function;

[0169] : The weight coefficient set for the j-th word in the vocabulary at the i-th position of the natural language navigation instruction sequence;

[0170] : The true probability of the jth word in the vocabulary at the i-th position in the true natural language navigation instruction sequence;

[0171] : The predicted probability of the jth word in the vocabulary at the i-th position in the natural language navigation instruction sequence predicted by the model;

[0172] : the length of the natural language navigation instruction sequence;

[0173] : Vocabulary size, that is, the total number of words or other language units that the model can use when generating natural language navigation instructions;

[0174] In one embodiment of the present invention, an artificial intelligence-based low-altitude navigation support system is provided, which is used to execute any of the artificial intelligence-based low-altitude navigation support methods described above.

[0175] The above describes an embodiment of the present invention, but this embodiment is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Ordinary technicians in this field can also make more forms of equivalent embodiments based on the inspiration of this embodiment, all of which are protected by this embodiment.

Claims

1. A low-altitude air navigation support method based on artificial intelligence, characterized in that: The following steps are involved: Step 100: Acquire aircraft position information, aircraft flight status information, aircraft surround view images, destination location information, surrounding terrain data, airspace control information, historical flight data, and historical navigation instruction data; Step 200: Based on the acquired information, a virtual navigation mark is set by using a deep reinforcement learning model to obtain a virtual navigation mark position set; The virtual navigation aid setting problem is modeled as a Markov decision process. The specific steps include: Step 201: Acquire destination location information, surrounding terrain data, airspace control information, and historical flight data; Step 202: Build a deep reinforcement learning model, define the state space S, the action space A, and design the reward function R; Step 203, setting a priority function for virtual navigation mark settings based on the airspace busyness and the proximity of the aircraft to other targets; Step 204 , calculating a priority value for each virtual navigation mark position in the historical flight data and integrating the obtained priority of each virtual navigation mark position into the state space S; Step 205: Initialize the deep reinforcement learning model and train the deep reinforcement learning model using historical flight data; Step 206: Generate a virtual navigation mark position set using the learned strategy based on the current state data; The state space S includes: aircraft position, aircraft flight status information, surrounding terrain data, and airspace control information; The action space A includes: combinations of various possible positions of the virtual navigation mark in the airspace; Each possible position in the airspace is discretized by constructing a three-dimensional spatial grid model to divide the airspace into a set of discrete coordinate points with a preset accuracy, and a digital storage system for the set of coordinate points is constructed based on a spatial database; Step 300: Based on the acquired information and the virtual navigation mark position set obtained by the deep reinforcement learning model, a deep convolutional neural network model is used to perform real-time rendering of the augmented reality scene to obtain an augmented reality navigation scene image; Step 400: Using the acquired information and the virtual navigation mark position set derived by the deep reinforcement learning model as input to the Seq2Seq model, outputting natural language navigation instructions; Step 500: Integrate augmented reality navigation scene images and natural language navigation instructions into the flight control interface to obtain pilot operation feedback.

2. The low-altitude air navigation support method based on artificial intelligence according to claim 1, characterized in that: The aircraft's surround view images need to be collected in real time through a multi-angle camera array and synchronized with the inertial measurement unit data to calibrate the image space coordinates.

3. The low-altitude air navigation support method based on artificial intelligence according to claim 1 is characterized in that: The specific steps of step 300 include: Step 301: Acquire aircraft position information, aircraft flight status information, virtual navigation mark position set, aircraft surround view image, surrounding terrain data, and airspace control information; Step 302: construct a deep convolutional neural network model; Step 303: Set the update frequency function of the airspace control information and update the airspace control information; In step 304, the acquired information is input into a deep convolutional neural network model, and after a series of feature extraction and information fusion operations, an augmented reality navigation scene image is generated.

4. The low-altitude air navigation support method based on artificial intelligence according to claim 1 is characterized in that: The specific steps of step 400 include: Step 401, obtaining aircraft position information, aircraft flight status information, virtual navigation mark position set, and historical navigation instruction data; Step 402: Build a Seq2Seq model. Step 403: Use historical data to train and learn the mapping relationship between state feature sequences and natural language navigation instructions; Step 404: Set an update frequency function for airspace control information and update the airspace control information. Step 405: Encode the acquired data into a feature vector sequence and input it into a Seq2Seq model, output and decode it to generate natural language navigation instructions.

5. The low-altitude air navigation support method based on artificial intelligence according to claim 1 is characterized in that: The pilot operation feedback in step 500 includes: command confirmation and command correction. If the feedback is a command correction, steps 100-500 are immediately re-executed.

6. The low-altitude air navigation support method based on artificial intelligence according to claim 1 is characterized in that: The reward function R is as follows: ； : Safety Reward; : efficiency reward; : Energy consumption penalty; : Regulatory compliance rewards; 、 、 、 : The first, second, third and fourth multi-objective weight coefficients; ； : The minimum real-time distance between the aircraft and obstacles; : safety distance threshold; : distance attenuation coefficient, the default value is 50; ； : The deviation distance between the aircraft's current position and the planned route; : the total length of the planned route; ； : The number of virtual navigation marks currently set; : The maneuvering frequency of the aircraft per unit time; 、 : The first and second energy consumption weight coefficients, the default values ​​are 0.1 and 0.5 respectively; ； : Number of violations of control rules; : Penalty coefficient of regulatory rules.

7. The low-altitude air navigation support method based on artificial intelligence according to claim 6 is characterized in that: The priority function is as follows: ； ； ； ； : airspace busy coefficient; : The real-time distance between the aircraft and the target point, i.e. the virtual beacon; : The first dynamic weight coefficient; : The second dynamic weight coefficient; : priority function; : The number of aircraft in the current airspace; : Maximum aircraft capacity of the airspace.

8. A low-altitude air navigation support system based on artificial intelligence, characterized in that: It is used to execute an artificial intelligence-based low-altitude navigation support method as described in any one of claims 1-7.

Images