Method for determining an emergency landing plan for an aerial drone, corresponding system and aircraft equipped with such a system

Abstract

The invention relates to a method for determining an emergency landing plan for an aerial drone, wherein: (2) a main neural network is applied to map data to determine at least one candidate area for an emergency landing runway; (3) an emergency landing runway is determined as the rectangle inscribed within a candidate area with the largest surface area; (4) each candidate area and intersecting or adjacent areas are formatted by polygonization; at least one additional piece of information relating to a candidate area is determined and stored with the formatted data; (5) predefined rules are applied via an inference engine to the formatted data and the additional data of each candidate area; and a candidate runway is declared valid if all the rules are verified. Figure for the abstract: Fig 4

Description

Title of the invention: Method for determining an emergency landing plan for an aerial drone, corresponding system and aircraft equipped with such a system Technical field

[0001] The technical field of the invention is the control of aerial drones, and more particularly the emergency landing of such drones. Previous techniques

[0002] To allow an emergency landing, emergency landing runways (acronym “PPU”) are defined throughout the trajectory defined in a flight plan. An emergency landing plan is defined as comprising at least one Emergency Landing Runway (referred to as “PPU” hereinafter) and at least one associated approach zone.

[0003] The choice of PPUs is made in advance during mission preparation. The choice of a PPU is complex because it must represent maximum safety for the aircraft and its environment.

[0004] Each PPU must therefore comply with numerous constraints. It must in fact have a minimum size to allow the drone to land, not present any obstacles during the approach phase, not require flying over urban areas or prohibited zones, not have a road axis at its ends while being accessible to be able to recover the drone.

[0005] Due to this complexity, in operational contexts, the preparation of PPUs is done manually by an operator preparing the mission, by selecting mapped areas. This work is based on the cartographic data of the area flown over.

[0006] Manually searching for PPUs has two major drawbacks. First, it is extremely time-consuming and tedious. Furthermore, if an unforeseen event prevents the homing of one or more pre-programmed PPUs, searching for one or more PPUs during flight, in real time, by an operator of the drone on a mission, may require more time than the drone has before having to land, which may be detrimental to flight safety.

[0007] The search for PPUs can also be based on the use of neural networks. Although advantageous, particularly in terms of time, it remains difficult to implement. Indeed, the use of a neural network can be difficult to understand for the operator ensuring the preparation of the mission or for the operator of the drone on mission, given the rigor required to guarantee the admissibility of the track. Indeed, neural networks rarely have perfect performance. They make mistakes. It is therefore necessary for an operator to understand what led the network to give a particular response. However, neural networks are known to be difficult to explain.

[0008] The invention aims to solve these technical problems. Statement of the invention

[0009] The subject of the invention is a method for determining an emergency landing plan for an aerial drone, in which: • a main neural network is applied to at least one piece of cartographic data, so as to determine at least one candidate area for an emergency landing strip, • for each candidate area for an emergency landing strip, at least one rectangle inscribed in said candidate area is determined, an emergency landing strip corresponding to the rectangle with the largest surface area, • each area candidate for an emergency landing runway and the areas intersecting or adjacent to each area candidate for an emergency landing runway are formatted by polygonization, • at least one additional piece of information relating to a candidate area for an emergency landing strip is determined, and the formatted data is stored along with the at least one additional piece of information, • predefined rules are applied via an inference engine on the formatted data and additional data of each candidate area for an emergency landing runway, • a candidate area for a valid emergency landing runway is declared if the said rules are verified, the said valid emergency landing runway being integrated into the emergency landing plan.

[0010] At least one candidate area can be determined by means of a determination of bounding boxes, a bounding box being determined for each candidate area.

[0011] At least one candidate area can be determined by means of a determination creation of a segmentation map.

[0012] At least one rule that must be respected by each emergency landing runway and / or by an approach zone linked to an emergency landing runway can be determined, as well as at least one corresponding parameter, said at least one rule that must be respected by each emergency landing runway and / or by an approach zone linked to an emergency landing runway is then applied via the inference engine, a candidate zone for an emergency landing runway is declared valid if said at least one rule that must be respected by each emergency landing runway and / or by an approach zone linked to an emergency landing runway are verified.

[0013] A candidate area for an emergency landing runway may be declared invalid if at least one rule is not verified, and the results of the rule verifications are stored for subsequent reverse verification.

[0014] The main neural network can be of the convolutional type, in particular Unet, ResNET, EfficientNet or Transformer.

[0015] When the rules applied by the inference engine involve areas neighboring the areas that are candidates for an emergency landing runway, information relating to at least one area neighboring a area that is a candidate for an emergency landing runway can be determined from the map data.

[0016] The aerial drone can move according to a predetermined flight zone, the cartographic data then being a function of the flight zone of the drone.

[0017] The cartographic data may include aerial or satellite image type data acquired prior to the flight or acquired by the drone's on-board camera.

[0018] The map data can be determined by an on-board camera.

[0019] An additional neural network of the semantic segmentation type can be applied to the images acquired by the on-board camera.

[0020] The training of the main neural network can be carried out via the expert system by producing a map of the candidate areas for an emergency landing runway determined by the main neural network, each candidate area being labeled as validated or invalidated by the expert system, said map being transmitted to the main neural network as training data, the main neural network thus improving its determination of the candidate areas for an emergency landing runway so that a greater portion of the candidate areas for an emergency landing runway are then considered admissible by the expert system.

[0021] The invention also relates to a system for determining an emergency landing plan for an aerial drone, the system comprising processing means, and at least one memory comprising a base of rules for determining an emergency landing track, a first base of facts and a second base of facts, the processing means being configured to carry out the method for determining an emergency landing plan as described above.

[0022] The system for determining an emergency landing plan may comprise processing means, and at least one memory comprising a base of rules for determining an emergency landing runway, a first base of facts and a second base of facts, the determination system also comprising means for acquiring an image of the ground, the processing means being configured to carry out the method for determining an emergency landing plan as described above.

[0023] Another object of the invention is an aircraft equipped with a system for determining an emergency landing plan as described above. Brief description of the drawings

[0024] Other aims, characteristics and advantages of the invention will appear on reading the following description, given solely by way of non-limiting example and made with reference to the appended drawings in which:

[0025] - [Fig.l] illustrates the generation of bounding boxes by a neural network,

[0026] - [Fig.2] illustrates the main elements of a PPU determination system embarked in a drone,

[0027] - [Fig.3] illustrates the main elements of a computing means configured for apply an inference engine,

[0028] - [Fig.4] illustrates the main steps of a method for determining a plan of installed urgently according to a first embodiment,

[0029] - [Fig.5] illustrates formatted map data in the vicinity of an area candidate for an emergency landing runway, and

[0030] - [Fig.6] illustrates the main steps of a method for determining a plan of installed urgently according to a second embodiment. Detailed description

[0031] To ensure the properties of speed and verifiability, the present invention uses an AI artificial intelligence approach combining a deep learning approach and an expert system approach in order to search for PPUs to define an emergency landing plan associated with a flight plan, or even PPUs outside a predetermined flight plan. This search can be implemented before the mission or during the flight from a ground station or on board.

[0032] To achieve this, a main neural network is applied to map information in order to determine candidate areas for a PPU. An expert system is implemented in order to confirm or deny candidate areas for a PPU, by manipulating a decision rule base.

[0033] The main neural network is preferably of the convolutional type. It may, for example, have an architecture of the U-Net, ResNET, EfficientNet or Transformer type.

[0034] Several types of input map data can be used to make this first determination by neural network. This can be, but is not limited to, one of the following information:

[0035] - An aerial or satellite image of the area,

[0036] - Land use map,

[0037] - Road network map,

[0038] - Slope map,

[0039] - Map of no-fly zones,

[0040] - Wind map...

[0041] - A combination of any of the above information.

[0042] The main neural network makes it possible to obtain as output a list of bounding boxes outlining the candidate areas for a PPU, or an image segmented at the pixel level indicating the candidate areas for a PPU and the non-candidate areas for a PPU depending on the neural network used.

[0043] Figure [Fig.l] illustrates the generation of such a bounding box BE by a neural network from input map data DNC.

[0044] The neural network may have one of the architectures described in the documents CHEN Zhao-Min, et al. “Hierarchical context embedding for region-based object detection”, European Conference on Computer Vision. Springer, Cham, 2020. p. 633-648, and STRUDEL Robin et al “Segmenter: Transformer for semantic segmentation”, Proceedings of the IEEE / CVF International Conference on Computer Vision. 2021. p. 7262-7272.

[0045] The neural network is trained on annotated training data, containing candidate areas and tracks annotated as admissible or inadmissible as PPUs, and representative of the context of use for the PPU search. The training data may, for example, be derived from manual annotation or from a method independent of the neural network such as that used by an expert system similar to that used to validate the inferences made by the neural network.

[0046] The use of the main neural network makes it possible to quickly obtain relevant proposals for candidate areas for a PPU. This technique perfectly meets the time constraint (mission preparation time or reaction time in the event of an unforeseen event). On the other hand, this approach has the following disadvantages:

[0047] - The use of a neural network is often seen as a black box, because it It is difficult to trace the decision criteria and provide a formal guarantee (in the context of flight safety), as neural networks are generally known to be difficult to explain,

[0048] - The use of a neural network can result in false detections, that is to say say declare as a candidate area for a PPU an area which should not be.

[0049] In a flight safety context, these two drawbacks are not acceptable. This is remedied by having an expert system validate the PPUs determined in the candidate areas for a PPU proposed by the neural network. main. The use of an expert system alone is subject to the polygonization of the data required by the expert system over the entire area of ​​interest, which can be lengthy in the context of a drone flight, particularly in the event of an emergency requiring the use of an emergency landing strip.

[0050] On the other hand, the joint use of an expert system and a neural network makes it possible to benefit from both the advantages of a neural network and the readability of the results of the expert system. The neural network saves time by eliminating the "vectorization" operation and extracting the information necessary for the application of the rules by the expert system. Only the candidate areas for a PPU declared admissible by the neural network are then processed by the expert system. The neural network thus provides a list of candidate areas for an emergency landing runway. Only these areas are vectorized in order to generate working data for the expert system. The reduction in the quantity of data to be vectorized also contributes to saving time.

[0051] An emergency landing strip is defined as a rectangle in which the drone can land. An emergency landing strip is determined as the largest area rectangle inscribed in a candidate area for a PPU proposed by the main neural network. Such a search for a rectangle allowing to maximize the area is notably described in the document Maximum-Area Rectangles in a Simple Polygon - Y.Choi et al. - arXiv:1910.08686vl [cs.CG] 19 Oct 2019.

[0052] When determining an emergency landing runway, the main axis of the emergency landing runway may be sought. Indeed, certain characteristics of an emergency landing runway and of the candidate area for a PPU supporting said emergency landing runway as well as the eligibility rules applied by the expert system possibly depend on this main axis. This is particularly the case for drones not having vertical landing capabilities.

[0053] For each emergency landing runway, at least one approach zone can be defined, possibly depending on the main axis and the direction of the emergency landing runway, or even two approach zones corresponding to two possible directions of approach for the same axis.

[0054] Each axis associated with its direction is associated with a set of rules coded in the expert system. For example, these rules represent constraints on the quality of the terrain (no steep slope) or its direct environment (no major road), etc.

[0055] Each given approach zone is associated with a set of rules given in the expert system. Several approach zones are necessary when it is desired to apply different rules in different approach zones, for example increasingly restrictive rules as the drone approaches the zone. candidate and that its altitude decreases.

[0056] Said expert system is an artificial intelligence algorithm capable of applying rules and producing results of application of the rules which are understandable by a human operator. In other words, this algorithm is therefore verifiable since the validation of an emergency landing runway is the result of the application of a set of rules and their logical combination. In the event of invalidation, it is possible for the system to show the choice rules which are not respected by an invalidated emergency landing runway.

[0057] Validation by the expert system is carried out by applying an inference engine to a first fact base based on a rule base.

[0058] The rule base contains the logical rules to be applied to the information characterizing the candidate area in order to declare it (or not) admissible as an emergency landing runway.

[0059] In the first fact base are stored the flight zone and all the data characterizing the zone, the emergency landing runway and the approach zones, as well as the data necessary for the application of the rules (input parameters) determining its admissibility.

[0060] A second fact base makes it possible to store intermediate data and the information necessary for the expert system rules to conclude on the admissibility of the emergency landing zone / runway, as well as the results of the analyses carried out.

[0061] The inference engine is configured to validate or invalidate the candidate PPUs associated with the candidate zones obtained from the main neural network stored in the first fact base by verifying compliance with the rules stored in the rule base by each candidate zone for an emergency landing runway, the emergency landing runway and possibly the approach zones.

[0062] The result of the verification of the stored rules and the results of the intermediate calculations are stored in the second fact base.

[0063] The rule base contains different rules, applicable to the area, the PPU and / or the approach areas, ranging from simple and unitary logical conditions (of the type IF... THEN...) to high-level conditions validating and invalidating the candidate runways for a PPU. For example, implementing rules such as:

[0064] IF land_occupation == Housing AND Parcel == APPROACH_ZONE THEN PPU <= KO

[0065] IF Rectangle == PPUCandidat AND Terrain_slope > 20% THEN PPU <= KO.

[0066] The rules for determining the validity of a candidate area for a landing runway emergency, included in the rule base, are, but are not limited to, the following:

[0067] - The emergency landing strip cannot be made up of a road axis zone,

[0068] - The emergency landing strip cannot be made up of an inhabited area,

[0069] - The emergency landing strip must have a minimum length, for example 800m (value which is defined according to the flight characteristics of the drone),

[0070] - The emergency landing strip must have a minimum width, for example 100m (value which is defined according to the flight characteristics of the drone),

[0071] - The emergency landing strip must not have a neighboring polygon comprising a high point having a height greater than a threshold, for example 10m within a predetermined distance (for example 100m) from its center,

[0072] - approach zone 1 (for example) must not contain any tall building greater than 25m,

[0073] - approach zone 2 must not contain more than 10 dwellings (or any other population density criterion),

[0074] - approach zone 3 is prohibited (e.g. nuclear power plant),...

[0075] Of course, such rules are in no way limiting so that one does not depart from the scope of the invention when the base contains other types of rules.

[0076] The first fact base contains the values ​​of the parameters called by the different rules and initialized before launching the complete analysis of the area of ​​interest, in particular the flight zone. These parameters are, for example, the minimum length of a runway or the flight zone. Other rules can also be used depending on the analysis of an area of ​​interest.

[0077] For example, the second fact base receives throughout the determination the confirmed facts concerning the different areas to be analyzed (potential areas) with the available characteristics, the candidate areas for an emergency landing strip validated as admissible and those inadmissible.

[0078] The available characteristics contain the information characterizing the candidate area as an emergency landing runway, the characteristics of the emergency landing runway, the characteristics of the approach areas and the information necessary for the application of the associated rules.

[0079] The inference engine makes it possible to automatically identify new facts, by logical deduction, by applying the different rules to the different known facts. These new facts are added to the second fact base. The addition of a new fact to the base is generally referred to by the Anglo-Saxon term "forward chaining / tracking".

[0080] The inference engine also makes it possible to answer questions asked by the user, for example, concerning the admissibility of the area as an admissible emergency landing runway, the reasons for non-admissibility, the length of the runway in the area. The addition of this information is generally referred to by the Anglo-Saxon term "backward chaining / tracking". This capability allows the operator to request explanations on the decision about a candidate runway. It therefore supports the explainability of the invention.

[0081] The PPU determination system comprises at least one processing means configured to carry out the PPU determination method and associated with a memory. In a particular embodiment, it comprises communication means, making it possible to consult remote cartographic databases or to transmit the PPUs to the drone. In another embodiment, the cartographic database is included in a memory of the determination system, or even embedded in the drone. In another embodiment, the system for determining an emergency landing plan is embedded in the drone as illustrated in figure [Fig.2]. In this case at least, the drone comprises an image acquisition means MAI and a processing means MT.

[0082] The image acquisition means MAI is configured to generate rasterized images of at least one camera view from the drone.

[0083] The processing means MT is configured to apply a neural network type method to cartographic data. In the case of processing on board the drone, the processing means MT is configured to apply a neural network type method to the images from the onboard camera.

[0084] It will be understood that depending on the origin of the data to be processed, the applied treatments differ. The data to be processed are essentially images acquired by the image acquisition means (e.g. an on-board camera), rasterized cartographic data (i.e. in the form of images formed by pixels), or vectorized cartographic data (i.e. in the form of images formed by vectors).

[0085] It will be understood that an acquired image is a rasterized image not including any additional information beyond the intensity and / or colorimetric components of each pixel.

[0086] Similarly, it will be understood that a rasterized cartographic image includes additional information relating to the occupation (plot, road, building, altitude, etc.) of each pixel, in addition to the intensity and / or colorimetric component information.

[0087] Finally, vectorized data includes information defining each point of the image as a vector relative to a common origin. Other vectors can then be defined by combining these vectors with a common origin. These other vectors can be associated with one or more data in order to form vectorized data. Examples of vectorized data are, for example, a vector representing a section of road, a vector representing the number of dwellings in a given area, etc.

[0088] By geometric construction, it is then possible to define each zone and plot as a polygon defined by the vectors of each of its vertices. Each polygon is then provided with the additional information of the pixels that compose it. For example, an approach zone can be defined by a vector describing its perimeter. It is possible to obtain the intersections of the road section vectors with the vector of the approach zone, or the vectors corresponding to the polygon of dwellings in the perimeter of the approach zone, etc. We then understand that a polygon is defined so that the pixels composing it present uniform additional information.

[0089] It will be understood that semantic segmentation allows the transition from a rasterized image to a rasterized cartographic image.

[0090] In addition, polygonization makes it possible to move from a rasterized cartographic image to vectorized cartographic data, without modifying the additional information.

[0091] It will also be understood that the inference engine can only work on vectorized data.

[0092] The processing means MT is also configured to carry out a method implementing an inference engine, configured so as to determine emergency landing tracks from the data stored in the first fact base and to determine compliance by each emergency landing track with the rules stored in the rule base and / or received from external sources by applying an inference engine.

[0093] The processing means MT stores the results of the determination of the PPUs, the result of the verification of the stored rules and the results of the intermediate calculations in the second fact base.

[0094] More precisely, the processing means MT comprises three calculation means or calculation units, referenced MCI, MC2 and MC3.

[0095] The first MCI calculation means is configured or programmed to apply a main neural network to rasterized cartographic data, to generate candidate areas for an emergency landing strip, in particular via bounding boxes and / or pixel segmentation of the candidate area, and to carry out the polygonization thereof.

[0096] The second calculation means MC2 is configured or programmed to carry out the polygonization of the segmented region with the largest intersection surface with each bounding box.

[0097] In another embodiment, the at least one parcel from the rasterized map data having the largest intersection with the at least one bounding box produced by the first network is extracted from the map database.

[0098] The third calculation means MC3 is configured or programmed to apply a geometric search algorithm in order to determine at least one candidate runway in each polygonized candidate area. This determination consists of searching for a rectangle included in the candidate area, of sufficient size to allow landing (a method such as Maximum-Area Rectangles in a Simple Polygon - Y.Choi et al. - 2019 may be used). The third calculation means MC3 is also configured or programmed to apply an inference engine to the candidate areas for a PPU and to the candidate runways based on data stored in the first and second bases, in fact and based on the rules stored in the rule base. The third calculation means MC3 validates or invalidates each candidate area based on the result of application of the rules.

[0099] Figure [Fig.3] illustrates in more detail the third calculation means MC3 in its part configured to apply the inference engine. It comprises means for applying an inference engine MI to data from the fact base BF according to rules from the rule base BR in conjunction with parameters from the parameter base BP. The fact base BF is enriched with external data, in particular from the determination of candidate zones and tracks on the cartographic data or from information requests. The result of the application of the inference engine MI is the determination of validated PPUs PPUv or invalidated zones ZI.

[0100] In a first embodiment, illustrated by the figure [Fig.4], the PPU determination method begins with a first step 1 during which the rules stored in the rule base that each PPU must comply with are defined. The defined and stored rules may be applicable to the area candidate for a PPU, to the PPU, or to at least one approach area. These rules are possibly parametric, with parameters that may differ depending on whether the rule is applied to the area candidate for a PPU, to the PPU, or to the at least one approach area. The parameters associated with these rules are then stored in the first fact base.

[0101] During a second step 2, at least one candidate area for an emergency landing strip is determined by means of a main neural network applied to at least one piece of cartographic data and it is stored in the second fact base.

[0102] If the main neural network produces candidate areas for a PPU resulting from a pixel segmentation, the polygonization of each area follows the external contour of the candidate areas.

[0103] If the main neural network produces candidate areas for an emergency landing track in the form of bounding boxes, it is necessary to polygonize the candidate areas for an emergency landing track before being able to apply an algorithm for searching for one or more candidate tracks.

[0104] To achieve this, we determine the plot with maximum intersection with the bounding box. This plot can come from a geographic information system or from an additional neural network of the semantic segmentation type applied to images acquired by the drone. In the latter case, the polygonization follows the external contour of the candidate area.

[0105] During a third step 3, at least one emergency landing runway is determined in each area that is a candidate for an emergency landing runway, by searching for a larger rectangle inscribed in the area that is a candidate for an emergency landing runway. According to the embodiments, the main axis and the landing direction of the emergency landing runway and / or at least one approach area can be determined.

[0106] During a fourth step 4, the data of the areas in the vicinity of the candidate areas for an emergency landing strip are formatted and this information is stored in the second fact base. Indeed, an expert system cannot reason directly on geographical data, even in the form of maps. The data of the neighboring, adjacent or intersecting areas of an area candidate for an emergency landing point are necessary so that the inference engine can apply the rules related to the respective position of these areas (for example, "is the area adjacent to the candidate area for an emergency landing strip a military zone?"). A step of preparing the data of these areas is necessary. The expert system can reason on polygons and their associated properties and relationships once put in an adequate data format (representation of a fact base for an expert system).

[0107] The properties used are, but are not limited to:

[0108] - Type: fields, meadow, water, dwelling, road, woods ....

[0109] - Width

[0110] -Length [YES] -Position

[0112] - Orientation of the plot

[0113] -Slope

[0114] - Wind direction

[0115] - average wind speed

[0116] Relations represent the relationships between polygons:

[0117] - Is adjacent to the east of polygon XX

[0118] - Is neighboring to the north of the polygon YY

[0119] - Is adjacent to the south of the polygon ZZ

[0120] - Is neighboring to the west of polygon VV

[0121] - Is located at a distance D from polygon XX (for D ranging from 0 to 5 km by example)

[0122] Figure [Fig.5] illustrates the formatted data in the vicinity of a candidate area for a PPU. A candidate area ZC has, for example, properties such as an oc cupation (field), a slope (1%), a width (60m), an orientation (North).

[0123] An adjacent area ZA located to the southeast of the candidate area and immediately adjacent has other properties such as occupation (dwellings), slope (3%), width (60m), orientation (North-East). Among the notable areas, a road R is represented. It also has properties such as slope (3%), width (10m) and orientation (North).

[0124] Similarly, the other zones are also associated with properties that can be subject to rule application.

[0125] This information can be extracted from available cartographic data (already vectorized data) or extracted manually (by photo-interpretation) or automatically by image processing algorithms as described in the publications “Satellite Image Semantic Segmentation”, E. Guérin et al. 2021, ArXiv (https: / / hal.archives-ouvertes.fr / hal-03375737).

[0126] The abstraction of information is done only for the polygons of the map data intersecting the emergency landing tracks proposed by the main neural network as well as the polygons located within a predetermined distance (for example, 1 km) from the emergency landing track. The predetermined distance depends on the type of drone.

[0127] During a fifth step 5, the rules of the rule base are applied via the inference engine to the data of the fact bases. For each area candidate for an emergency landing runway, it is then determined whether the rules of the rule base are respected. If this is the case, the area candidate for an emergency landing runway is declared valid. If at least one rule is not respected, the area candidate for an emergency landing runway is declared invalid. In all cases, the results of the rule verifications are stored for a subsequent reverse verification.

[0128] Due to its optimized structure, the determination of emergency landing runways as described above can be carried out at any time, even during flight, from a ground station.

[0129] In a second embodiment illustrated by the figure [Fig.6], the method for determining PPUs is carried out by calculation means on board an aerial drone. Indeed, if the drone leaves its flight zone significantly, it may be useful to be able to recalculate the PPUs on board because it may not be possible for the drone to reach the pre-programmed PPUs or to communicate with a ground station to receive updated PPUs.

[0130] Without communication with a ground station, the map information also cannot be updated based on deviation from the flight zone.

[0131] In such a case, the method begins with a preliminary step 1a, during which one performs an acquisition of at least one image by an on-board camera. A semantic segmentation of the at least one acquired image is then carried out by an additional neural network, in particular as described in one of the documents "Satellite Image Semantic Segmentation", E. Guérin et al LONG, Jonathan, et al. "Fully convo-lutional networks for semantic segmentation." Proceedings of the IEEE conference on computer vision and pattern recognition. 2015 or STRUDEL, Robin, et al. “Segment: Transformer for semantic segmentation”, Proceedings of the IEEE / CVF International Conference on Computer Vision. 2021. p. 7262-7272.

[0132] The additional neural network is previously trained on aerial images representative of images acquired by a camera in flight.

[0133] The method then continues with steps 1 to 5 similar to the steps described above with regard to the first embodiment.

[0134] During a first step 1, the rules applicable to each candidate zone for an emergency landing runway, an emergency landing runway and / or at least one approach zone are defined. The parameters associated with the rules are also defined, when this is the case, for each of the candidate zones for an emergency landing runway, the emergency landing runways and / or the at least one approach zone. The rules are stored in a rule base and the parameters, when there are any, are recorded in the first fact base.

[0135] During a second step 2, at least one candidate area for a PPU is determined by means of a main neural network applied to the at least one acquired image processed by the second neural network. The candidate areas for an emergency landing track are stored in the second fact base.

[0136] If the main neural network produces candidate areas for an emergency landing runway in the form of a pixel segmentation, each area is polygonized by following the external contour of the candidate areas. The geometric search algorithm for one or more candidate runs can then be applied to these candidate areas.

[0137] If the main neural network produces candidate areas for an emergency landing runway in the form of bounding boxes, for each box, the parcel (i.e., an area for which the segmentation is uniform) with a maximum intersection with the bounding box is determined. This parcel is derived from the data determined by the additional neural network of the semantic segmentation type applied to the images acquired by the drone. The polygonization then follows the external contour of this parcel.

[0138] The vertices of the polygons obtained are expressed by geometric transformation in a reference geographic frame using knowledge of the position and pose of the camera (calculation carried out using the drone navigation data and the intrinsic parameters of the camera). The camera pose is known, for example, from the position of the camera's shooting axis and from the camera's viewing angles in the aircraft's reference frame. In other words, a "back-projection" algorithm is applied which transforms data seen in the camera's plane into data projected onto the ground. The resulting data is therefore comparable to cartographic data. By "back-projection", we mean a set of matrix calculation steps based on the drone's position, its altitude and the camera's viewing angles. The rest of the processing is done with these transformed polygons.

[0139] During a third step 3, at least one emergency landing runway is determined in each area that is a candidate for an emergency landing runway, by searching for a larger rectangle inscribed in the area that is a candidate for an emergency landing runway. According to the embodiments, the main axis and the direction of the emergency landing runway and / or at least one approach area can be determined.

[0140] Objects that can be manipulated by the inference engine are constructed by associating complementary information with each polygon.

[0141] Additional information is for example:

[0142] a. The class (house, lake, forest, field, road, etc.),

[0143] b. The length and width of the polygon (for the emergency landing strip polygon, the largest inscribed rectangle will be considered),

[0144] c. The geographic coordinates of the center of the polygon,

[0145] d. Relations with neighboring polygons (“is north of”, “is neighbor of” ...),

[0146] e. The characteristic data associated with the candidate area, the landing strip emergency, and possibly at least one approach zone, these characteristic data being necessary for the application of the rules by the inference engine.

[0147] Then a step similar to the fifth step 5 of the first embodiment is carried out during which the rules of the rule base are applied via the inference engine to the data of the fact bases. For each area candidate for an emergency landing runway, it is then determined whether the rules of the rule base are respected. If this is the case, the area candidate for an emergency landing runway is declared valid. If at least one rule is not respected, the area candidate for an emergency landing runway is declared invalid. In all cases, the results of the rule checks are stored for a subsequent reverse check (“backward chaining”).

[0148] In a particular embodiment, the expert system is used to carry out the training of the main or additional neural networks. The candidate areas for an emergency landing runway determined by the main neural network are processed by the expert system which scores them as admissible or not one after the other. the other, by applying the same system of rules applied to characterize the admissibility of a candidate emergency landing zone or runway in the inference phase of new candidate zones / runways.

Claims

Claims

1. Method for determining an emergency landing plan for an aerial drone, in which: - a main neural network is applied to at least one cartographic data, so as to determine at least one candidate zone for an emergency landing runway, - for each candidate zone for an emergency landing runway, at least one rectangle inscribed in said candidate zone is determined, an emergency landing runway corresponding to the rectangle having the largest surface area, - each candidate zone for an emergency landing runway and the intersecting or adjacent zones of each candidate zone for an emergency landing runway are formatted by polygonization, - at least one additional information relating to a candidate zone for an emergency landing runway is determined, and the formatted data are stored accompanied by the at least one additional information,- predefined rules are applied via an inference engine to the formatted data and additional data of each candidate area for an emergency landing runway, - a candidate area for a valid emergency landing runway is declared valid if said rules are verified, said valid emergency landing runway being integrated into the emergency landing plan.,

2. A determination method according to claim 1, wherein at least one candidate area is determined by means of a determination of bounding boxes, a bounding box being determined for each candidate area.

3. A determination method according to claim 1, wherein at least one candidate area is determined by determining a segmentation map.

4. Determination method according to any one of claims 1 to 3, in which at least one rule is determined to be respected by each emergency landing runway and / or by an approach zone linked to an emergency landing runway, as well as at least one corresponding parameter, then said at least one rule is applied to be respected by each emergency landing runway and / or by an approach zone linked to an emergency landing runway via the inference engine, declares a candidate area for an emergency landing runway valid if the at least one rule to be complied with by each emergency landing runway and / or by an approach area linked to an emergency landing runway is verified.

5. A determination method according to any one of claims 1 to 4, wherein a candidate area for an emergency landing runway is declared invalid if at least one rule is not verified, and the results of the rule verifications are stored for subsequent reverse verification.

6. Determination method according to any one of claims 1 to 5, in which the main neural network is of the convolutional type, in particular Unet, ResNET, EfficientNet or Transformer.

7. A determination method according to any one of claims 1 to 6, wherein when the rules applied by the inference engine involve areas neighboring the areas that are candidates for an emergency landing runway, information relating to at least one area neighboring a area that is a candidate for an emergency landing runway is determined from the map data.

8. Determination method according to any one of claims 1 to 7, in which the aerial drone moves according to a predetermined flight zone, the cartographic data then being a function of the flight zone of the drone.

9. Determination method according to any one of claims 1 to 8, in which the cartographic data comprises aerial or satellite image type data acquired prior to the flight or acquired by the drone's on-board camera.

10. A determination method according to any one of claims 1 to 8, wherein the map data is determined by an on-board camera.

11. Determination method according to claim 10, in which an additional neural network of the semantic segmentation type is applied to the images acquired by the on-board camera.

12. A determination method according to any one of claims 9 to 11, wherein the training of the main neural network is carried out via the expert system by producing a map of the candidate areas for an emergency landing runway determined by the main neural network, each candidate area being labeled as validated or invalidated by the expert system, said map being transmitted to the main neural network as training data, the main neural network thus improving its determination of candidate areas for an emergency landing strip so that a greater proportion of candidate areas for an emergency landing strip are subsequently considered admissible by the expert system.

13. System for determining an emergency landing plan for an aerial drone, the system comprising processing means, and at least one memory comprising a rule base for determining an emergency landing track, a first fact base and a second fact base, the processing means being configured to carry out the method for determining an emergency landing plan according to any one of claims 1 to 8.

14. System for determining an emergency landing plan for an aerial drone, the system comprising processing means, and at least one memory comprising a rule base for determining an emergency landing track, a first fact base and a second fact base, the determination system also comprising means for acquiring an image of the ground, the processing means being configured to carry out the method for determining an emergency landing plan according to any one of claims 9 to 12.

15. Aircraft equipped with a system for determining an emergency landing plan according to any one of claims 13 or 14.

Images