IMPROVED METHOD FOR ASSISTING WITH THE LANDING OF AN AIRCRAFT AND SYSTEM CONFIGURED TO EXECUTE THE METHOD.

The method and system use image processing and PnP algorithms to enhance runway alignment accuracy, addressing the challenge of positioning aircraft on multiple runways, particularly in poor visibility, by confirming correct alignment with the intended runway.

FR3149416A1Pending Publication Date: 2024-12-06AIRBUS OPERATIONS (SAS)
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
FR2023005454
Authority / Receiving Office
FR · FR
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-31
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

Current aircraft landing systems struggle to accurately position an aircraft relative to multiple runways in airports, especially in degraded visibility conditions without automatic landing systems, leading to potential confusion and safety risks.

Method used

A method and system that utilize image processing and a PnP algorithm to detect and reproject runway features, calculate theoretical aircraft positions, and assess coherence scores to confirm the correct runway alignment using a geodetic database and a front camera on the aircraft.

Benefits of technology

Enhances the accuracy of runway alignment confirmation, reducing the risk of confusion and ensuring safe landings by providing reliable position estimation and verification of the aircraft's alignment with the intended runway.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

The invention relates to a method for assisting the landing of an aircraft (1) adapted to determine as many probable theoretical positions (POS1, POS2, POS3) of the aircraft facing a plurality of runways (100a, 100b, 100c) of a destination airport facility (100); to operate reprojections (102a, 102b, 102c) of said runways in an image (F) obtained by a camera (10c) of the aircraft (1) by means of a theoretical camera model (e.g., "pinhole camera model"), each with reference to one of the determined theoretical positions (POS1, POS2, POS3); and to establish for each of the projections operated a consistency score enabling the subsequent provision of an estimated position (POSA) of the aircraft (1). Advantageously, this allows us to confirm or refute whether the runway seen from the camera (10c) of the aircraft (1), and positioned facing the aircraft (1), is indeed the intended destination runway (100b) for landing. Fig. 1
Need to check novelty before this filing date? Find Prior Art

Description

Title of the invention: IMPROVED METHOD FOR ASSISTANCE IN THE LANDING OF AN AIRCRAFT AND SYSTEM CONFIGURED TO CARRY OUT THE METHOD. Technical field

[0001] The present invention relates to a method for assisting the landing of an aircraft on a runway of an airport facility. The invention relates more particularly to a method for determining the position of the aircraft with respect to a plurality of runways of the same airport facility, from a shot taken from the aircraft, then verifying the consistency of the determined position. The invention further relates to a system configured to execute the aforementioned landing assistance method. STATE OF THE PRIOR ART

[0002] Current aircraft include numerous pilot assistance devices or systems, including, for example, systems for assisting in carrying out an optimized descent towards a destination airport facility or systems aimed at avoiding confusion upon arrival at the destination between a destination landing runway and a taxiway parallel to this runway. Furthermore, not all airports are equipped with systems allowing automatic landing (e.g. ILS).Patent application FR3122408A1 relates to an automated system on board an aircraft automatically comparing the regions obtained from a reprojection potentially containing airport runways in a shot taken from an aircraft with a synthetically reconstructed image, in the same spatial reference as that of the shot, of a destination runway, from coordinates of characteristic points of these runways of the destination airport obtained from a database. This makes it possible to automatically identify all the runways of the destination airport independently of any other global positioning system of the aircraft (e.g. GPS).

[0003] Many airport facilities comprise several runways, sometimes arranged parallel to each other, and a risk of confusion may arise in poor visibility conditions, and when there are no other positioning means available to position an aircraft relative to the landing runway automatically. There is then a need to confirm good positioning of an arriving aircraft, operating a final approach to a destination runway of an airport facility.

[0004] The situation can be improved. Statement of the invention

[0005] An object of the present invention is to propose a method for assisting the landing of an aircraft making it possible to confirm correct positioning of the aircraft relative to a destination landing runway of an airport facility, in particular when the latter comprises a plurality of landing runways, as well as an automated landing assistance system configured to execute such a method.

[0006] To this end, a method is proposed for assisting the landing of an aircraft on a runway of a destination airport facility, the method comprising:

[0007] - detecting a plurality of landing strips in an image representative of a shooting taken from the aircraft,

[0008] - selecting a detected landing runway from among the plurality of runways landing strips detected in the image, and detect characteristic points of this runway in said image,

[0009] - determine as many theoretical positions of the aircraft as there are runways landing present in the destination airport facility from the characteristic points of the landing runway selected in the image, an algorithm for determining a position of the aircraft relative to said runway from the shot, of the “PnP” type, and the coordinates of the characteristic points of each of the runways in a geodetic database, then, for each of the determined theoretical positions of the aircraft: • carry out a reprojection of each of said landing strips in the image using the determined theoretical position, the coordinates of the geodetic database and the algorithm for determining a position of the aircraft relative to said strip from the shot, of the “PnP” type, • establish and memorize a consistency score between the characteristic points of the tracks detected in the image and the characteristic points of the tracks reprojected in the image, then

[0010] - provide information representative of an estimated position of the aircraft from the memorized consistency scores.

[0011] Advantageously, it is thus possible to confirm or deny a good position of the aircraft facing a presumed destination landing strip among a plurality of landing strips.

[0012] According to one embodiment, the step of selecting a landing runway comprises determining a landing runway detected as being the most centered among the runways in the image or the one with the highest detection index.

[0013] According to one embodiment, establishing a consistency (or correlation) score between the characteristic points of the runways detected in the image and the characteristic points of the runways reprojected in the image comprises establishing a consistency index for each pair of runways comprising a landing runway detected in the image and a landing runway reprojected in the image from a distance between a barycenter of the runway detected in the image and a barycenter of the runway reprojected in the image, then determining the consistency score from the indices.

[0014] According to one embodiment, the consistency score is the average of the indices.

[0015] The invention also relates to a landing assistance device comprising electronic circuitry configured to:

[0016] - detecting a plurality of landing strips in an image representative of a shooting taken from the aircraft,

[0017] - selecting a detected landing runway from among the plurality of runways landing strips detected in the image, and determine characteristic points of this runway in said image,

[0018] - determine as many theoretical positions of the aircraft as there are runways landing in the destination airport facility from the characteristic points of the landing runway selected in the image, an algorithm for determining a position of the aircraft relative to said runway from the shot, of the “PnP” type, and the coordinates of the characteristic points of each of the runways in a geodetic database, then, for each of the determined theoretical positions of the aircraft:

[0019] - performing a reprojection of each of said landing strips in the image in using the theoretical position, the coordinates of the geodetic database and the algorithm for determining a position of the aircraft relative to said runway from the shot, of the “PnP” type,

[0020] - establish and store a consistency score between said characteristic points of said tracks detected in said image and the characteristic points of the tracks reprojected in the image, then,

[0021] - provide information representative of an estimated position of the aircraft from the memorized consistency scores.

[0022] According to one embodiment, the electronic circuitry configured to establish a consistency score between the characteristic points of the tracks detected in the image and the characteristic points of the tracks reprojected in the image comprises electronic circuitry configured to establish a consistency index for each pair of tracks comprising a landing track detected in the image and a track landing reprojected into the image, from a distance between a barycenter of the runway detected in the image and a barycenter of the runway reprojected into the image, then, determine the consistency score from the indices.

[0023] The invention also relates to an aircraft comprising a landing assistance device as previously described.

[0024] The invention finally relates to a computer program product comprising program code instructions for executing the steps of a landing assistance method as previously described as well as a storage medium comprising a computer program product as mentioned above. Brief description of the drawings

[0025] The above-mentioned features of the invention, as well as others, will appear more clearly on reading the following description of an exemplary embodiment, said description being made in relation to the attached drawings:

[0026] [Fig.l] schematically and symbolically illustrates an aircraft comprising a landing assistance system according to an embodiment presented facing a destination airport installation;

[0027] [Fig.2] schematically illustrates a reprojection of a landing strip in a image plane from a front camera of an aircraft;

[0028] [Fig.3] schematically illustrates a landing assistance method according to a embodiment;

[0029] [Fig.4] schematically illustrates a shot of three landing strips of a same airport installation from a front camera of an aircraft;

[0030] [Fig.5] schematically illustrates the three landing strips already shown on the [Fig.4] in an image provided by the front camera that took the shot in [Fig.4];

[0031] [Fig.6] schematically illustrates a detection of landing strip type objects identified by bounding boxes in the image already shown in [Fig.5];

[0032] [Fig.7] schematically illustrates a reprojection of the runways of the aero installation port already shown in [Fig.l] in the image already illustrated in [Fig.5] and [Fig.6], from a first theoretical position determined by the aircraft;

[0033] [Fig.8] schematically illustrates a reprojection of the runways of the airport facility already shown in [Fig.l] in the image already illustrated in [Fig.5] and [Fig.6], from a second determined theoretical position of the aircraft;

[0034] [Fig.9] schematically illustrates a reprojection of the runways of the aero installation port already shown in [Fig.l] in the image already illustrated in [Fig.5] and [Fig.6], from a third theoretical position determined by the aircraft; and,

[0035] [Fig. 10] schematically illustrates an internal architecture of a device landing assistance according to one embodiment.

[0036] DETAILED DESCRIPTION OF EMBODIMENTS

[0037] [Fig.l] schematically and symbolically represents an aircraft 1 comprising a landing assistance system 10 configured to execute a landing assistance method according to an embodiment of the invention. The system 10 is notably configured to determine a position of the aircraft 1 relative to a destination landing runway 100b on which a landing of the aircraft 1 is planned imminently. The destination landing runway 100b is a runway of an airport facility 100. The landing runway 100b of the airport facility 100 is elongated and rectangular in shape. Advantageously, the destination landing runway 100b is referenced in a database RWYDB directly or indirectly accessible by the landing assistance system 10 of the aircraft 1.According to one embodiment, the RWYDB database is downloaded into the assistance system 10 of the aircraft 1, for example, prior to a flight. According to one embodiment, the RWYDB database comprises, for each of the airport facility runways listed therein, coordinates of characteristic points of the runway (corners and midpoints of the runway thresholds, for example) according to a WGS84 world geodetic format (from the English “World Geodesy System”). According to one embodiment, the landing assistance system 10 is configured to communicate with a remote server, for example a database server installed in a ground station, which comprises the RWYDB database of airport facility runways. Advantageously, the aircraft 1 is equipped with at least one front camera 10c (shown in [Fig.2]), which is configured to take pictures of a sector called the “front sector” also called the “field of vision” or “FOV” from the English “Field Of Vision”, towards which the aircraft is heading when it is flying. Preferably, the front camera 10c is arranged in the radome of the aircraft 1. According to variants, the front camera 10c is arranged on any part of the fuselage of the aircraft 1 provided that its position is compatible with taking pictures of the front sector, which sector is also generally visible from the cockpit of the aircraft 1. The front camera 10c operates like a conventional image capture device, that is to say that it is configured to take successive pictures and provide successive images F each representative of a view of the environment facing the aircraft 1, from the aircraft 1, at the time of the picture taking.According to one embodiment, the front camera 10c is for example configured to take 30 shots per second and to provide 30 images F per second to the landing assistance device 10. Advantageously, the camera 10c or the landing assistance system 10 comprises a module for detecting regions of interest configured to. detect one or more apparent regions of interest during a shot, from one of the corresponding images F. In other words, the region of interest detection module is a module comprising electronic circuitry configured to detect one or more objects (regions of interest) of a given type in an image F obtained from the front camera 10c. According to the example described, the detected object(s) are landing strips present in the field of vision of the camera 10c. According to one embodiment, the object detection module comprises a software or hardware implementation of a deep artificial neural network or DCNN (Deep Convolutional Neural Network). Such a DCNN module may consist of a set of numerous artificial neurons, of the convolutional type or of the perceptron type, and organized by successive layers connected to each other.Such a DCNN module is classically inspired by a simplistic model of the functioning of a human brain where many biological neurons are connected to each other by axons. For example, a module called YOLOv4 (from the English acronym "You Only Look Once version 4") is a DCNN type module which allows object detection in images, and which is called "one-stage", that is to say whose architecture is composed of a single module of combined proposals of rectangles framing objects ("bounding boxes" or "bounding boxes", from the English) and classes of objects in the image.In addition to the artificial neurons previously described, the YOLOv4 module uses functions known to those skilled in the art such as batch normalization, dropblock regularization, weighted residual connections between layers, or a non-maximal suppression step that eliminates redundant proposals for detected objects.According to one embodiment, the object detection module has the possibility of predicting a list of objects present in one or more images of one or more video contents by providing, for each object, a rectangle framing the object in the form of coordinates of points defining the rectangle in the image, the type or class of the object from a predefined list of classes defined during a learning phase, and a detection score representing a degree of confidence in the detection thus carried out. Advantageously, the object detection module operating from an image F is configured to extract coordinates of characteristic points of each of the landing strips detected in an image F, with reference to the image plane (or image reference) of the image F.For example, the object detection module of aircraft 1 is capable of providing coordinates of the four corners of a landing strip detected in an image F, as well as two points defining a longitudinal central axis of the runway concerned. The . image plane of the front camera 10c is defined here as a plane perpendicular to the optical axis of the front camera 10c and an image provided by the camera 10c therefore comprises a projection of all the elements seen by the camera in this reference image plane.

[0038] Cleverly and advantageously, it is then possible, for any landing strip whose characteristic points are referenced in a landing strip geolocation database, to use the coordinates present in the database or even coordinates of points determined from the coordinates present in the database, as well as the coordinates of the same points of the runway as detected in an image F obtained from the front camera, to define a position of the aircraft at the time of the shot from which the image F originates, thanks to the use of an algorithm for determining a position of the aircraft relative to said runway from the shot, of the type "Perspective-n-Point" (from English) also commonly called "PnP" algorithm or reprojection algorithm. Such an algorithm is also called pose calculation algorithm. The term "pose calculation" here designates a determination of a position of an aircraft with respect to a spatial reference frame, which can include its position but also its orientation, its attitude (positioning of its axes in space). It is an algorithm aimed at determining in particular the relative position according to six degrees of freedom of a camera (or more precisely of its optical center), positioned on its optical axis of vision, in relation to an object considered in space, and vice versa, from a set of correspondences of the points Mi referenced in a spatial reference frame X, Y, Z and their respective projections mi in an image plane obtained from the camera.Thus, for example, an algorithm for determining a position of the aircraft relative to said runway from the shot, of the PnP type, can define a position of the camera 10c from the geodesic coordinates of the corners A, B, C, D of the runway 100a and the coordinates in the image F of the points a, b, c, d which are the representation of the points A, B, C and D in the image F provided by the front camera 10c.

[0039] [Fig. 2] illustrates a relative positioning of the aircraft 1 facing the destination landing strip 100b whose four corners are identified and referenced A, B, C, D. [Fig. 2] further illustrates an image F as provided by the front camera 10c of the aircraft 1 and in which appears a representation of the runway 100b, detected by the object detection module of the aircraft 1 configured to detect one or more landing strips in the image F provided by the front camera 10c. The image F comprises a representation a, b, c, d in the reference frame u, v of coordinates of the points of the image F, of the characteristic points A, B, C and D which are the corners of the landing strip 100b in the real world. According to one embodiment, geodesic coordinates xA, yA, zA; xB, yB, zB; xC, yC, zC and xD, yD, zD respec tively associated with points A, B, C, D of runway 100b in the “real world” are known and recorded in the RWYDB runway database with reference to a spatial reference frame X, Y, Z.

[0040] According to one embodiment, the landing assistance system 10 of the aircraft 1 is configured to determine a first position of the optical center lOf of the camera 10c, and therefore consequently of the aircraft 1 relative to the landing runway 100b using a first algorithm, of the PnP algorithm type. The PnP algorithm executed according to the invention, also called here algorithm for determining a relative position of the aircraft relative to a destination runway, or algorithm for calculating pose relative to the destination runway, is not detailed since the numerous calculation operations carried out according to such an algorithm are known and its detailed description is not useful for understanding the invention.

[0041] The use of a PnP algorithm, for determining a position of the aircraft relative to said runway from the shot, therefore makes it possible, from a representation of a landing runway selected in the image F, and knowing the geodesic coordinates of the characteristic points A, B, C and D of each of the landing runways 100a, 100b and 100c, to determine as many theoretical positions of the aircraft 1 facing the airport facility 100 as there are runways present in the airport facility 100. Thus, if one of the runways seen by the front camera 10c of the aircraft 1 does indeed belong to the airport facility 100, at least one of the theoretical positions determined using the PnP type algorithm will be exact (within calculation errors).According to the example described, the landing runways 100a, 100b and 100c of the airport facility 100 are three in number and it is therefore possible to determine three theoretical positions of the aircraft 1 which are named here POS1, POS2 and POS3.

[0042] According to one embodiment, the selection of a runway detected in the image F (or more precisely the selection of a representation of a determined runway in the image F) is carried out by choosing the runway which appears the most centered in the image F, insofar as it is considered that, except in special cases, the radio navigation means used are such as to allow positioning of the aircraft on arrival facing the destination runway and in the axis of the destination runway, which has the consequence of making the destination runway appear centrally in the image F representative of the field of vision of the front camera 10c. Obviously, this example is not limiting and another runway representation could be used.

[0043] [Fig. 3] is a flowchart illustrating steps of a method for assisting the landing of the aircraft 1 executed by the landing assistance system 10.

[0044] An initial step S0 is a step at the end of which all of the means implemented by the method are correctly initialized and normally operational. Furthermore, at the end of step SO, the aircraft 1 is sufficiently close to the destination airport facility for several runways of the airport facility 100 to be present in the field of vision of the camera 10c. According to the example described, the three runways 100a, 100b and 100c of the airport facility 100 are visible at the end of step S0. The object detection module of the aircraft 1 then operates, during a step SI, a detection of each of the runways present in the field of vision of the camera 10c, so as to be able to identify them by bounding boxes in the image F provided by the front camera 10c. The theoretical positions PO1, POS2 and POS3 are then determined during a step S2, from a landing runway selected in the image F. According to the example described, the selected runway is the most centered runway in the image, namely the landing runway 100b.In another example, the selected landing strip is the one for which a confidence index provided by the object detection module in the image is the largest (an index associated with a bounding box or "boundix box").

[0045] According to yet another example, the step of selecting a landing runway comprises determining an identifier of the runway (such as for example a runway number painted on its runway threshold according to its orientation). A determination of the destination runway is possible even if the detected runway is not the destination runway because the relative position of all the runways of the airport facility is known. According to the example described, the theoretical position POS1 is determined from the geodetic coordinates of the runway 100a in the database RWYDB, the theoretical position POS2 is determined from the geodetic coordinates of the runway 100b in the database RWYDB, and the theoretical position POS3 is determined from the geodetic coordinates of the runway 100c in the database RWYDB.Once the theoretical positions POS1, POS2 and POS3 have been determined by the execution of a PnP algorithm, steps S3 and S4 are successively carried out as many times as there are theoretical positions determined (therefore here three times according to the example described), that is to say as many times as there are landing runways present in the airport facility 100. A reprojection of the runways 100a, 100b and 100c is then carried out during step S3, for each of the theoretical positions POS1, POS and POS3 determined so as to determine a level of consistency, during a step of step S4, between the landing runways 100a, 100b and 100c detected as present in the image F, and the landing runways 100a, 100b and 100c as reprojected in the image F using the use of a theoretical camera model (e.g. a camera of a type called a "pinhole camera model" from English).In step S4, the level of consistency between the detected landing strips and the reprojected landing strips is quantified by calculating a consistency score for each of the previously determined theoretical positions. Dif . Various methods can be used to determine a consistency score. For example, if a reprojected track appears entirely within a bounding box of predetermined size defined around a detection bounding box, or centered on it, a partial score (also called here "index") determined for this track is positive and if the reprojected track does not appear entirely within a predetermined bounding box, but only partially, the partial score for this track is zero, and finally, if the reprojected track appears only outside a predetermined bounding box, the partial score for this track is negative.According to another exemplary embodiment, establishing a consistency score between the characteristic points a, b, c, d of the runways 100a, 100b and 100c detected in the image F and the equivalent characteristic points of the runways reprojected in the image F comprises establishing a consistency index for each pair of runways comprising a landing runway detected in the image F and a landing runway reprojected in the image F from a distance between a geometric barycenter of the runway detected in the image F and a geometric barycenter of the runway reprojected in the image F, then subsequently determining the consistency score from the indices previously calculated for each of the runways. Other variants are of course possible since the calculation of a consistency score is representative of an overall level of consistency between the runways as reprojected and the runways as detected.In the example described, a consistency score SC is determined from indices SCI, SC2 and SC3 respectively associated with pairs of detected and reprojected tracks.

[0046] A step S5 aims, after each successive iteration of steps S3 and S4, to determine whether a consistency score SC has been determined for all the runways of the airport facility 100 or in other words for all the determinable theoretical positions (here POS1, POS2 and POS3) of the aircraft 1 facing the airport facility 100, using a theoretical camera model (e.g. “pinhole camera model” for the reprojections).

[0047] If a consistency score SC has been determined for all the tracks in step S5, the method then provides, in a step S6, an estimated position information POSA of the aircraft from the consistency scores SC respectively determined for the different theoretical positions POS1, POS2 and POS3. Otherwise, the method loops back to step S3 in order to carry out a new iteration of steps S3 and S4 for a position (and therefore a track) not yet processed by these steps. According to one embodiment, the final estimated position information POSA is the theoretical position for which the highest consistency score SC is obtained among the consistency scores SC associated with the theoretical positions POS1, POS2 and POS3 and stored. Indeed, at least one of the determined theoretical positions POS1, POS2 and POS3 is the position of the aircraft at the time of the shot generating the image F since, unless there is an error in the identification of the airport facility visible from the front camera, the landing runway detected in the image F and then selected to determine the theoretical positions POS1, POS2 and POS3 actually corresponds to one of the runways of the airport facility 100. The hypothesis of an error in the identification of the airport facility visible from the aircraft is unlikely since radio navigation means or visual references could have been used to operate an approach of the aircraft towards the destination airport facility 100. [Fig.4], [Fig.5], [Fig.6], [Fig.7], [Fig.8] and [Fig.9] are intended to illustrate the detection and reprojection operations carried out during the execution of the method described above in relation to the flowchart of [Fig.3].

[0048] According to an alternative embodiment, a reprojection of each of said landing runways is carried out in the image F using the determined theoretical position, the coordinates of the geodetic database and a theoretical model (or modeling) of the camera such as for example a "pinhole camera model", from the English, then a coherence score (or correlation) is established and stored between the region of the image F in which the runways are located obtained from a reprojection using the theoretical camera model once the theoretical position of the aircraft has been determined, and reconstructed synthetic images of the corresponding runways of the airport facility, then information representative of an estimated position of the aircraft is provided from the stored coherence scores.

[0049] [Fig. 4] schematically illustrates the field of view (or FOV) seen from the front camera of the aircraft 1 on final approach to the airport facility 100. The landing runways 100a, 100b and 100c are therefore visible in the field of view of the camera 10c, centered along a vision axis of the camera 10c. More broadly, other elements of the airport facility 100 are visible in the field of view of the camera 10c, but are not shown in the figures, for the sake of simplification. Such elements not shown are, for example, airport buildings and halls, one or more control towers as well as equipment inherent in the operation and service of an airport facility (emergency centers, roads, telecommunications equipment, radars, etc.), as well as aircraft and land vehicles.

[0050] [Fig. 5] represents an image F as resulting from a shot taken by the front camera 10c and then supplied by the latter. Thus the image F comprises a representation of the runways present in the field of vision, as long as the visibility conditions between the aircraft 1 and each of the landing runways allow it. In the case where the visibility conditions do not allow a representation of all the runways present in the field of vision of the camera 10c, for example due to a local cloudiness, the runway selected for determining as many theoretical positions of the aircraft as there are runways identified in the airport facility 100 will be done by selecting a runway visible in image F. It is therefore entirely possible to establish the position of the aircraft in relation to the destination runway even if the latter is not visible, but other runways of the airport are.

[0051] Obviously, if no track is visible in image F, the method cannot be executed.

[0052] [Fig.6] illustrates the result of an object detection module performing a detection of landing strip type objects from the image F. Bounding boxes 101a, 101b and 101c make it possible to identify respectively in the image F the landing strips 100a, 100b and 100c detected.

[0053] [Fig.7], [Fig.8] and [Fig.9] illustrate reprojections of the tracks 100a, 100b and 100c carried out in the image F by successively considering the different theoretical positions POS2, POS3 and POS1 of the aircraft 1.

[0054] [Fig.7] illustrates a reprojection of the tracks 100a, 100b and 100c from the theoretical position POS2 previously determined in step S2. The track representation 102a is the reprojection of the track 100a; the track representation 102b is the reprojection of the track 100b and the track representation 102c is the reprojection of the track 100c. According to one embodiment, the greater the coherence, the higher the determined coherence score SC and vice versa.

[0055] It is possible to observe a good overall consistency of the reprojections 102a, 102b and 102c of tracks with the detected tracks 100a, 100b and 100c and the consistency score SC determined for this theoretical position POS2 is high.

[0056] [Fig.8] illustrates a reprojection of the tracks 100a, 100b and 100c from the theoretical position POS3 previously determined in step S2. The track representation 102a is the reprojection of the track 100a; the track representation 102b is the reprojection of the track 100b and the track representation 102c is the reprojection of the track 100c. It is possible to observe a very poor overall consistency of the track reprojections 102a, 102b and 102c with the detected tracks 100a, 100b and 100c and the consistency score SC determined for this theoretical position POS3 in step S4 is therefore much lower than that determined for the theoretical position POS2.

[0057] [Fig. 9] illustrates a reprojection of the tracks 100a, 100b and 100c from the theoretical position POS1 previously determined in step S2. The track representation 102a is the reprojection of the track 100a; the track representation 102b is the reprojection of the track 100b and the track representation 102c is the reprojection of the track 100c. Just as for the situation described above in relation to [Fig. 8], it is possible to observe in [Fig. 9] a very poor overall consistency of the track reprojections 102a, 102b and 102c with the detected tracks 100a, 100b and 100c and the consistency score SC determined at step S4 for this theoretical position POS1 is also therefore lower than that determined for the theoretical position POS2.

[0058] Thus, the landing assistance system is cleverly configured to determine an estimated position POSA of the aircraft and to ultimately confirm or deny the correct position of the aircraft facing the destination landing runway 100b planned for a landing.

[0059] [Fig. 10] is a schematic representation of an example of internal architecture of the landing assistance system 10, also called here landing assistance device 10. Let us consider for illustration purposes that [Fig. 10] illustrates an internal arrangement of the landing assistance system 10 as installed in the aircraft 1. It is noted that [Fig. 10] could also schematically illustrate an example of hardware architecture of the landing assistance system 10 according to a configuration where the landing assistance system 10 is external to the aircraft 1 and communicates with the latter via one or more wireless communication links.

[0060] According to the example of hardware architecture shown in [Fig. 10], the landing assistance system 10 then comprises, connected by a communication bus 19: a processor or CPU (“Central Processing Unit” in English) 11; a RAM (“Random Access Memory” in English) 12; a ROM (“Read Only Memory” in English) 13; a storage unit such as a hard disk (or a storage media reader, such as an SD (“Secure Digital” in English) card reader) 14; a communication interface module 15 allowing the landing assistance system 10 to communicate with remote devices, such as other on-board systems of the aircraft 1 including the front camera 10c, a database server, or even such as the aircraft 1, if applicable.

[0061] The processor 11 of the landing assistance system 10 is capable of executing instructions loaded into the RAM 12 from the ROM 13, from an external memory (not shown), from a storage medium (such as an SD card), or from a communication network. When the landing assistance system 10 is powered on, the processor 11 is capable of reading instructions from the RAM 12 and executing them. These instructions form a computer program causing the implementation, by the processor 11, of all or part of a landing assistance method described in relation to [Fig. 3] or of described variants of this method.

[0062] All or part of the method described in relation to [Fig. 3] or its described variants can be implemented in software form by executing a set of instructions by a programmable machine, for example a DSP (“Digital Signal Processor” in English) or a microcontroller, or be implemented in hardware form by a machine or a dedicated component, for example an FPGA (“Field-Programmable Gate Array” in English) or an ASIC (“Application-Specific Integrated Circuit” in English). In general, the landing assistance system 10 comprises electronic circuitry configured to implement the methods described in relation to itself. Obviously, the landing assistance system 10 further comprises all the elements usually present in a system comprising a control unit and its peripherals, such as a power supply circuit, a power supply supervision circuit, one or more clock circuits, a reset circuit, input-output ports, interrupt inputs, bus drivers, this list being non-exhaustive.

Claims

Claims

1. Method for assisting the landing of an aircraft (1) on a landing strip (100b) of a destination airport facility (100), said method comprising: - detecting a plurality of landing strips (100a, 100b, 100c) in an image (F) representative of a shot taken from the aircraft (1). - selecting a detected landing strip (100b) from among said plurality of detected landing strips (100a, 100b, 100c) in said image (F), and determining characteristic points (a, b, c, d) of this strip in said image (F), - determining as many theoretical positions (POS1, POS2, POS3) of the aircraft (1) as there are landing runways (100a, 100b, 100c) in said destination airport facility (100) from the characteristic points (a, b, c, d) of the selected landing runway (100b) in said image (F), from an algorithm for determining a position of the aircraft relative to said runway from the shot (F), of the “PnP” type, and from the coordinates of said characteristic points (A, B, C, D) of each of the runways (100a, 100b, 100c) in a geodetic database (RWYDB), then, for each of the determined theoretical positions (POS1, POS2, POS3) of the aircraft: • performing a reprojection (102a, 102b, 102c) of each of said landing runways (100a, 100b, 100c) in said image (F) using said theoretical position (POS1, POS2, POS3), said coordinates of said geodetic database (RWYDB) and said algorithm for determining a position of the aircraft relative to said runway from the shot, of the “PnP” type, • establishing and storing a consistency score (SC) between said characteristic points (a, b, c, d) of said detected tracks in said image (F) and said characteristic points of said reprojected tracks (102a, 102b, 102c) in said image (F), then - provide information (POSA) representative of a position estimated aircraft from stored consistency scores.

2. A method of assisting the landing of an aircraft (1) according to claim 1, wherein the step of selecting a landing runway (100b) comprises determining a landing runway detected as being the most centered among said landing runways (100a, 100b, 100c) in said image (F) or that having a highest detection index.

3. A method for assisting the landing of an aircraft (1) according to one of claims 1 and 2, wherein establishing a consistency score (SC) between said characteristic points (a, b, c, d) of said detected runways in said image (F) and said characteristic points of said reprojected runways (102a, 102b, 102c) in said image (F) comprises establishing a consistency index for each pair of runways comprising a detected landing runway (100a, 100b, 100c) in said image (F) and a reprojected landing runway (102a, 102b, 102c) in said image (F) from a distance a barycenter of said detected runway in said image F and a barycenter of said reprojected runway in said image (F), then, determining said consistency score from said indices.

4. A method of assisting the landing of an aircraft (1) according to claim 3, wherein said consistency score is the average of said indices.

5. Landing assistance device (10) comprising electronic circuitry configured to: detect a plurality of landing strips (100a, 100b, 100c) in an image (F) representative of a shot taken from the aircraft (1) and determine runway characteristic points (a, b, c, d) for each of said runways (100a, 100b, 100c) detected in said image (F), - select a detected landing strip (100b) from said plurality of detected landing strips (100a, 100b, 100c) in said image (F), - determine as many theoretical positions (POS1, POS2, POS3) of the aircraft (1) as there are landing strips (100a, 100b, 100c) in said destination airport facility (100) from the characteristic points (a, b, c, d) of the selected landing strip (100b) in said image (F), an algorithm for determining a position of the aircraft relative to said runway from the “PnP” type image capture and the coordinates of said characteristic points (A, B, C, D) of each of the runways (100a, 100b, 100c) in a geodetic database (RWYDB), then, for each of the determined theoretical positions (POS1, POS2, POS3) of the aircraft: • performing a reprojection (102a, 102b, 102c) of each of said landing runways (100a, 100b, 100c) in said image (F) using said theoretical position (POS1, POS2, POS3), said coordinates of said geodetic database (RWYDB) and said algorithm for determining a position of the aircraft relative to said runway from the “PnP” type image capture, • establish and memorize a consistency score (SC) between said characteristic points (a, b, c,d) said tracks detected in said image (F) and said characteristic points of said tracks reprojected in said image, then - provide information (POSA) representative of an estimated position of the aircraft from the stored consistency scores.,

6. An aircraft landing assistance device (1) according to claim 5, wherein the electronic circuitry configured to establish a consistency score (SC) between said characteristic points (a, b, c, d) of said detected tracks in said image (F) and said characteristic points of said reprojected tracks (102a, 102b, 102c) in said image (F) comprises electronic circuitry configured to establish a consistency index for each pair of tracks comprising a detected landing track (100a, 100b, 100c) in said image (F) and a reprojected landing track (102a, 102b, 102c) in said image (F) from a distance a barycenter of said detected track in said image F and a barycenter of said reprojected track in said image (F), then, determine said consistency score from of said indices.

7. Device for assisting the landing of an aircraft (1) according to one of the

8.

9.

10. Claims 5 and 6, wherein establishing a consistency score (SC) between said characteristic points (a, b, c, d) of said detected runways in said image (F) and said characteristic points of said reprojected runways (102a, 102b, 102c) in said image (F) comprises establishing a consistency index for each pair of runways comprising a detected landing runway (100a, 100b, 100c) in said image (F) and a reprojected landing runway (102a, 102b, 102c) in said image (F) from a distance a barycenter of said detected runway in said image F and a barycenter of said reprojected runway in said image (F), then, determining said consistency score from said indices. Aircraft comprising a landing assistance device (10) according to one of Claims 5 to 7. Computer program product comprising program code instructions for executing the steps of a method according to one of claims 1 to 4 when said instructions are executed by a processor of a device for assisting the landing of an aircraft Storage device comprising a computer program product according to claim 9.

Citation Information

Patent Citations

  • AIRPORT APPROACH ASSISTANCE SYSTEM AND PROCEDURE

    FR3122408A1

  • Automated landing area detection for aircraft

    US20100039294A1

  • Method and system for automatically updating at least one airport database

    US20200285828A1

  • Landing site localization for dynamic control of an aircraft toward a landing site

    US20210319709A1

  • Airport approach assistance system and method

    US20220348353A1