Probabilistic decision of tracked object state to ensure accurate runway intrusion determination

Abstract

The invention relates to probabilistic decision-making of tracked object state to ensure accurate runway intrusion determination. A system and method for implementing a high confidence threshold when it is determined that a runway is unobstructed without obstacles. A landing aircraft periodically scans a runway area to confirm, track and propagate trajectories from reported and non-collaborative objects. Invocation objects that appear as very small artifacts are less likely to be detected in each sensor frame. The system and method propagates a trajectory incurring an object to confirm that the object will smooth the runway prior to landing of the aircraft by tracking the object and using a probabilistic decision maker.

Description

Technical Field

[0001] This disclosure relates to airport runway incursion detection. Background Technology

[0002] Of the many phases of flight, the landing phase is not optional. This is a period of high workload for pilots and high risk for aircraft. Incursions on runways designated for landing are relatively rare. However, the potential consequences when an incursion occurs can be catastrophic. Airport runway incursions are currently primarily addressed by humans (e.g., but not limited to pilots and air traffic controllers). This assessment is based on, for example, visual cues, radio communications, and situational understanding, and is conducted through pilot and air traffic controller training and an understanding of when an incursion might occur. Categories of potential targets include vehicles (including aircraft, rotorcraft, and ground vehicles), humans, and wildlife. The challenges of detecting incursions include the need for high-speed assessments and a dynamic environment, including moving detection platforms (aircraft), moving intruders, and changing seasons and airports. In 2021, the Federal Aviation Administration (FAA) reported 1,627 runway incursion incidents at 5,217 airports in the United States. In the first quarter of 2023, seven different U.S. airlines experienced seven runway incursions.

[0003] Existing solutions attempt to make decisions using end-to-end machine learning models. While the output of such models can be evaluated and their performance compared to probabilistic requirements, it is currently impossible to build probabilistic guarantees into these models. There are also concerns about whether such models will continue to meet probabilistic requirements in new environments.

[0004] A system and method are needed to integrate low-probability detections over time to ensure that probabilistic requirements are met for either missed runway incursions (false negative probability) or reported absence of an incursion (false positive probability). Furthermore, it addresses the problem of predicting future object trajectories, enabling the differentiation between objects currently on the runway (but not causing an incursion), ongoing incursions requiring a go-around to prevent a potential collision, and those that can be cleared in time for a safe landing.

[0005] In autonomous systems, a method for tracking objects and a decision-maker for visual inspection systems are needed. The proposed method can be adopted by aircraft and automotive companies seeking autonomous capabilities based on camera systems. What is needed is a system that can be used with autonomous aircraft systems or to provide human pilots with warning systems for runway incursions to ensure safe landings. What is needed is a system that can be used on current and future aircraft. Summary of the Invention

[0006] A system of one or more computers can be configured to perform specific operations or actions by installing software, firmware, hardware, or combinations thereof on the system, which, in operation, causes the system to perform actions. A system of one or more computer programs can be configured to perform specific operations or actions by including instructions that, when executed by a data processing device, cause the device to perform actions. One general aspect includes a method for determining the probability of runway incursion during aircraft landing. The method includes integrating object detections over time by detecting possible new and existing object tracks, assigning object detections to object tracks, and filtering object detections. The method includes estimating object states based on the integrated object detections, predicting future object movement and trajectory previews based on the estimated object states, dynamic models, and track history, determining the probability of runway incursion based on the predicted future object movement, and adapting the detections over time to new environments based on the accuracy of the probability. Other embodiments of this aspect include corresponding computer systems, apparatuses, and computer programs recorded on one or more computer storage devices, each computer system, apparatus, and computer program configured to perform actions of the method.

[0007] Implementation methods may include one or more of the following features. Detecting possible new and existing object tracks may include accessing track history, sensor measurements associated with possible new and existing object tracks, and track previews. The method may include detecting and tracking non-cooperative objects in a pre-selected area during aircraft landing, and determining the probability of runway incursion based on visual detection probabilities, the number of scans of the pre-selected area associated with aircraft landing, the number of updates to detected existing and new object tracks, a pre-selected false negation threshold, and a pre-selected false affirmation threshold. Statistical gating includes a rectangular gate based on state covariance to filter object detections considered for allocation, wherein an auction algorithm is configured to use distances between non-cooperative objects for allocation, and wherein object detections not used to maintain existing tracks are allocated to new trial tracks. The pre-selected area is based on the boundaries of the airport runway, the boundaries of the taxiway system associated with the airport runway, and the boundaries of the runway safety area associated with the airport runway. The pre-selected false negation threshold may include a pre-selected maximum allowable value for false negations. The pre-selected false affirmation threshold may include the maximum cumulative probability of false affirmations. The method may include: declaring a runway incursion when the ratio of scan counts to update counts reaches a threshold based on a pre-selected false negation threshold and a pre-selected false affirmation threshold. The method may include filtering the trajectory of a non-cooperative object based on track history, sensor measurements, and a trajectory preview generated by a dynamic model. Filtering may include: estimating the trajectory preview of the non-cooperative object by providing the track history to a first Kalman filter; estimating the acceleration of the non-cooperative object by providing the trajectory preview and sensor measurements to a second Kalman filter; and smoothing the trajectory of the non-cooperative object based on providing the track history, sensor measurements, and estimated acceleration to a Gaussian process. The method may include: receiving the sensor measurements; estimating the acceleration of possible new and existing objects; determining the trajectory preview based on the estimated acceleration; and generating a covariance estimate of the predicted state and the object trajectory. Estimating the acceleration of possible new and existing objects may include: determining the trajectory preview using a modified Kalman filter. The method may include smoothing the object trajectory and obtaining an inference. Smoothing the object trajectory may include feeding the object trajectory, sensor measurements, and trajectory preview to a Gaussian process. Implementations of the described technology may include hardware, methods or processes, or computer software on a computer-accessible medium.

[0008] One general aspect includes a computer system for determining the probability of runway incursion during aircraft landing. The computer system includes a hardware processor and a non-volatile storage medium storing instructions that, when executed by the hardware processor, perform operations that may include: detecting and tracking non-cooperative objects in a pre-selected area during aircraft landing; and determining the probability of runway incursion based on a visual detection probability, the number of scans of the pre-selected area associated with aircraft landing, the number of updates to the tracks of detected existing and new objects, a pre-selected false negative threshold, and a pre-selected false positive threshold. Other embodiments of this aspect include corresponding computer systems, apparatuses, and computer programs recorded on one or more computer storage devices, each computer system, apparatus, and computer program configured to perform actions of the method.

[0009] Implementations may include one or more of the following features. The computer system may include filtering the tracks of non-cooperative objects based on track history, sensor measurements, and track previews generated by a dynamic model. Filtering may include: estimating the track preview of the non-cooperative object by providing tracking history to a first Kalman filter; estimating the acceleration of the non-cooperative object by providing the track preview and sensor measurements to a second Kalman filter; smoothing the track of the non-cooperative object based on providing track history, sensor measurements, and estimated acceleration to a Gaussian process; integrating object detections over time by detecting possible new and existing object tracks, assigning object detections to object tracks, and filtering object detections; estimating object states based on the integrated object detections; predicting future object movement and track previews based on the estimated object states, the dynamic model, and track history; determining the probability of runway incursion based on the predicted future object movement; and adapting detection over time to new environments based on the accuracy of the probability. Implementations of the described techniques may include hardware, methods, or processes, or computer software on a computer-accessible medium.

[0010] One general aspect includes a computer program product for determining the probability of runway incursion during aircraft landing. The computer program product includes instructions for performing operations including: detecting and tracking non-cooperative objects in a pre-selected area during aircraft landing; determining the probability of runway incursion based on visual detection probability, the number of scans of the pre-selected area associated with aircraft landing, the number of updates to detected existing and new object tracks, a pre-selected false negation threshold, and a pre-selected false affirmation threshold. The operations include filtering the tracks of non-cooperative objects based on track history, sensor measurements, and a track preview generated by a dynamic model. Filtering may include: estimating the track preview of the non-cooperative object by providing the tracking history to a first Kalman filter; estimating the acceleration of the non-cooperative object by providing the track preview and sensor measurements to a second Kalman filter; and smoothing the tracking of the non-cooperative object based on providing the tracking history, sensor measurements, and estimated acceleration to a Gaussian process. The operations may also include integrating object detections over time by detecting possible new and existing object tracks, assigning object detections to object tracks, and filtering object detections. The operation may further include estimating object state based on integrated object detection, predicting future object movement and trajectory preview based on the estimated object state, dynamic model, and track history, and determining the probability of runway incursion based on the predicted future object movement. The operation may also include adapting detection to new environments over time based on probability accuracy. Other embodiments of this aspect include corresponding computer systems, apparatuses, and computer programs recorded on one or more computer storage devices, each computer system, apparatus, and computer program configured to perform actions of the method.

[0011] It should be understood that the foregoing general description and the following detailed description are merely exemplary and explanatory, and do not limit the claimed invention. Attached Figure Description

[0012] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate various aspects of the invention and, together with the specification, serve to explain the principles of the invention.

[0013] Figure 1 This is a schematic block diagram of the architecture of a combined tracking and decision-making system according to embodiments of the present disclosure;

[0014] Figure 2 This is a schematic block diagram illustrating the components of a tracking system according to an embodiment of the present disclosure;

[0015] Figure 3 This is a schematic block diagram of a track filter module according to an embodiment of the present disclosure;

[0016] Figure 4This is a graphical comparison between a conventional Kalman filter and a track filter according to an embodiment of the present disclosure;

[0017] Figure 5 This is a graphic example of four consecutive scans of the same airport segment (example requires numbering);

[0018] Figure 6 The graphical output of the system and method according to embodiments of this disclosure, the left side of the runway being unobstructed, and the right side of the runway being intruded; and

[0019] Figure 7 This is a flowchart of a method for determining runway incursions according to an embodiment of this disclosure.

[0020] It should be noted that some details in the accompanying drawings have been simplified and drawn for ease of understanding, rather than maintaining strict structural accuracy, detail, and proportion. Detailed Implementation

[0021] Reference will now be made in detail to the invention, examples of which are shown in the accompanying drawings. In the drawings, the same reference numerals are consistently used to denote the same elements. In the following description, reference is made to the accompanying drawings, which form a part thereof, and specific examples of practicing the invention are illustrated therein. Therefore, the following description is merely exemplary.

[0022] Now for reference Figure 1 For a safe landing on a runway, it is crucial to check for the presence of objects in the runway area. System 100 according to embodiments of this disclosure includes a trajectory system 101 and a decision system 103, which provide warnings to the pilot or signals to an autonomous planning and control module. Detection systems for runway landings often fail to see detected objects because the size of objects in the runway area perceived from the aircraft collecting visual data is quite small. The trajectory system 101 and decision system 103 (1) integrate detections over time, (2) improve object state estimation, and (3) predict future object movement.

[0023] The term "runway" is used herein to include a defined area for aircraft landing and takeoff, a taxiway for ground movement of aircraft / taxiing, and areas such as blast mats and oversized zones. The surface of a runway can be a natural material (e.g., soil, water, or ice) or another type of material (e.g., asphalt or concrete). Runways can include water surfaces, strips adjacent to larger runways used for aircraft landing training, vertical takeoff and landing (VTOL) airports, or helicopter landing pads. Aircraft can include, but are not limited to, flying vehicles, such as, but not limited to, commercial aircraft, private aircraft, military aircraft, ships, and helicopters, as well as other types of flying and hovering vehicles. Runway dimensions vary; for example, a small runway can be 800 feet long and 26 feet wide, or a large runway can be 40,000 feet long and 900 feet wide.

[0024] Visual runways may or may not have visual markings. Non-precision instrument runways may include visual markings such as centerlines for horizontal positioning guidance, aiming points for vertical positioning guidance, and buoys. Precision instrument runways include burst pads and over-limit zones, start and end designated space markers, centerlines, aiming points, buoys, and other horizontal and vertical approach guidance reference points. For example, single runways, parallel runways, intersecting runways, and open V-runways exist.

[0025] Now for reference Figure 2 In some configurations, the track system 101 includes, but is not limited to, a track manager 201, a track correlator 203, a track filter 205, and a self-reporting fusion unit 207. The track manager 201 maintains an internal track database of current and past object states and provides an interface layer to external system components. The track manager 201 also coordinates communication between internal subsystems and provides homography functionality capable of compensating for camera movement. The track correlator 203 uses, for example but not limited to, statistical gating and auction algorithms to assign detections to existing and new tracks. In some configurations, when using statistical gating, a rectangular gate based on state covariance is used for existing tracks. In some configurations, an auction algorithm using the distance between objects to process the actual assignment is used to filter detections considered for assignment. Object detections not used to maintain existing tracks are assigned to new trial tracks for future consideration. An updated list of object detections is provided to the track filter 205 (in...). Figure 3 (As described in the text), the track filter 205 returns a smooth state estimate. Self-reported detections are provided to a self-reported fusion system, which provides a list of fused tracks to the track manager layer. Based on the detection and tracking of non-cooperative objects on the runway area, the decision system 103 ( Figure 1 ) uses detector 105 ( Figure 1 The probability of detection is used to estimate runway incursions.

[0026] Now for reference Figure 3 The inputs to the track filter 205 include, but are not limited to, historical track data, sensor measurements 309, and a track preview. In some configurations, the track preview is generated by a Gaussian process 307 with a dynamic model. In some configurations, the track filter 205 includes, but is not limited to, two Kalman filters 301 / 305 and a Gaussian process 307. The first Kalman filter 301 receives the sensor measurements 309 and provides an acceleration estimate using a predictive model within the Kalman filter. The second Kalman filter 305 also receives the sensor measurements 309 and an acceleration estimate of the object. The estimate is used for the preview. The second Kalman filter 305 also provides a hazard state estimate between historical and current measurements. The track from the history, the current sensor measurements, and the preview are fed into the Gaussian process 307 to smooth the track and obtain inferences.

[0027] Continue to refer to Figure 3 The preview dynamic model of a discrete-time system is:

[0028] x k|k+1 =A k x k +B k u k y k =C k x k (1),

[0029] x k =[x 1,k ,x 2,k ] T ,

[0030] Among them, u k It is the input; y k It is the output, i.e., the measured value; and the system matrix is:

[0031]

[0032] The modified Kalman filter according to embodiments of this disclosure is designed to estimate the object acceleration and generate a predicted state as follows. The predicted state and covariance estimate are:

[0033]

[0034] The estimated state is:

[0035]

[0036] The system matrices Fk and Hk, and the noise covariance Qk are:

[0037]

[0038] The update process for the state and covariance is as follows. The innovation and its covariance are:

[0039]

[0040] Among them, the measurement covariance R k yes The optimal Kalman gain is:

[0041]

[0042] The updated state and covariance estimates are:

[0043]

[0044] The fitting residual after measurement is:

[0045]

[0046] To generate additional states between the current measurement and the preview, the Kalman filter prediction is used as:

[0047]

[0048] To obtain a preview, accelerate the estimation. The data is fed into the dynamic model in equation (1). In some configurations, a method according to embodiments of this disclosure is designed in a discrete-time system. The estimation in equation (1) is obtained by iteratively applying the following method. Preview of dynamic model 303 generation:

[0049]

[0050] The number of iterations can be selected based on the application. In some configurations, four preview x... k+2 …x k+6 It is generated through four iterations of equation (11).

[0051] A second Kalman filter, 305, is used to obtain the history based on the filter delay. The predicted states and covariance of the Kalman filter are:

[0052]

[0053] Among them, the system and measurement noise covariance yes:

[0054]

[0055] Now for reference Figure 4The diagram shows a comparison between result 401 obtained using a conventional Kalman filter and result 403 obtained using a track filter according to an embodiment of this disclosure. Line 405, representing the velocities from two objects, is not well aligned with the taxiway and indicates a noisy output, while line 407, representing the velocities from two other objects, is well aligned with the taxiway and indicates a smooth output.

[0056] Refer again Figure 3 Predicted state Current measured value z k Additional states And preview x k+2 …x k+4 The data is fed into Gaussian process 307. By using history, sensor measurements 309, and a preview, Gaussian process 307 treats historical patterns as machine learning algorithms, and provides information about the new environment and adaptation to the new environment through online estimation and the preview trajectory generated by dynamic model 303, as well as the uncertainty of the predefined dynamic model of Gaussian process 307, which enables the estimation of the accuracy of probability calculations.

[0057] The method according to embodiments of this disclosure utilizes information from dynamic model 303 (as shown in equation (1)) and acceleration estimation in equation (8). Using Gaussian process 307. For Gaussian process 307, the linear time-varying stochastic differential equation is given:

[0058]

[0059] Wherein, state x(t) and input u g (t) is:

[0060] x = [x1, y1, x2, y2] T , u g (t) = [u x ,u y ] T (15),

[0061] x1 and y1 are positions; x2 and y2 are velocities. The system matrices A(t), B(t), and N(t) are:

[0062]

[0063] The process noise w(t) is represented by the (symmetric, positive-definite) power spectral density matrix Q. c And δ are given by a (stationary) zero-mean Gaussian process:

[0064]

[0065] δ is the Dirac delta function. The solution to equation (14) is:

[0066]

[0067] Where Φ(t,s) is the following transition matrix:

[0068]

[0069] In order to obtain prior information, the mean of equation (14) is:

[0070]

[0071] Since the mean is zero, the last term is averaged with respect to w(t) in equation (14). For non-cooperative objects, the acceleration input u... g (t) is unavailable. The modified Kalman filter in equation (8) is used to obtain the estimate. It is used in Gaussian processes. The estimated value is used. Vector average Written as:

[0072]

[0073] in,

[0074]

[0075] The covariance function can be calculated based on the following definition:

[0076]

[0077] The covariance components between two different times are:

[0078]

[0079] Where, for i = 1…M and

[0080]

[0081] Based on Bayes' theorem, the maximum a posteriori (MAP) trajectory is:

[0082]

[0083] The prior P(x) of the Gaussian process is:

[0084]

[0085] Likelihood P(D(x) i )|x) is:

[0086]

[0087] To transform the nonlinear optimization into a linear optimization, the state x and the measurement function g(x) are approximated as:

[0088]

[0089] Using equation (30), the nonlinear optimization problem in equation (27) is transformed into:

[0090]

[0091] After solving the linear system in equation (31), iterate Until the convergence criterion is met. The linear optimization in equation (31) is rewritten as:

[0092]

[0093] Among them, Q in equation (26) i and Based on the dynamics estimated in equation (21), the sparse matrix structure, i.e. It's a piece with three opposite corners.

[0094] Decision System 103 ( Figure 1 The decision system 103 employs a probabilistic strategy combined with online adjustments of various trajectory parameters to ensure that probabilistic requirements are met. Figure 1 The system contains prior information about the boundaries of the airport runway and taxiway system, as well as the Runway Safe Area (RSA) containing the designated landing runway. The system determines the maximum permissible cumulative probability of false negations. The maximum permissible cumulative probability is a system requirement value. It is used to derive other system parameters to ensure system performance meets requirements. False negations occur when an object is present within the RSA but not being tracked, potentially leading to collisions. Determining the maximum permissible cumulative probability includes tracking track birth, track death, the associated gate size of the trial track, the number of required blank space observations, and detection slice allocation. A measured statistic is the maximum permissible cumulative probability of false affirmations. False affirmations occur when a track is initialized on an object that does not actually exist. From a safety perspective, it is important to minimize their frequency, as unnecessary turnarounds reduce the airport's operational safety margin. Potential consequences also include reduced efficiency, flight delays, and loss of pilot confidence. The scan-to-track update ratio is used to determine when a track is declared an intrusion.

[0095] Now for reference Figure 5 In addition to the track system 101 ( Figure 1 In addition to providing online updates, the decision-making system 103 ( Figure 1The system maintains a map of the current number of looks for each airport segment. This is maintained as multiple polynomial contours updated with each scan. Slices overlapping existing contours can result in an increase in the total number of looks for that segment. The edges of each contour shrink between sensor scans because previously undetected dynamic objects may have entered the contour area. This functionality is integrated with the Tracking System 101 (… Figure 1 They work together to obtain the maximum cumulative probability of false negations. In some configurations, decision system 103 ( Figure 1 Three different results are reported to the pilot. The first result is insufficient evidence. This indicates that the system has not integrated enough scans to meet the probability criteria shown in areas 503, 505, and 507. It is possible that the "air-view" polynomial does not encompass the entire RSA (false negation requirement). It can also indicate that the track within the RSA does not have a sufficiently high scan-track update ratio (false affirmative criterion). Additional scans allow the decision to become a definitive result. The result is Runway Clear 501, meaning there is no track within the runway safety area and that the segment has been scanned pre-selected a number of times to meet the false negation criterion, or an intrusion is in progress, indicating that the track is predicted to be within the RSA when the aircraft lands.

[0096] Now for reference Figure 6 This image shows two side-by-side views used for situational awareness assessment during flight testing. The image represents a real-time camera view with overlapping slices and object detection. When intrusion is possible, markers provide the identification of the target, the percentage of the Runway Safe Area (RSA) currently covered by a pre-selected number of scans, and the hysteresis after the current time. Rectangles indicate objects in the context of the scanned area. Boxes 601 and 651 are areas where a pre-selected number of scans have occurred (referred to herein as pre-selected areas). Boxes 603 and 653 are projections onto the slices in a two-dimensional top-down view. Boxes 609 and 659 are projections from the runway view of the aircraft. Points 607 and 657 are objects that may cause intrusion. For objects that are being tracked but are not expected to cause intrusion, different graphics can be used to distinguish them from intrusive objects. The object represented by point 607 is not expected to cause intrusion, while the object represented by point 657 is expected to cause intrusion.

[0097] Now for reference Figure 7The method 700 for determining the probability of runway incursion during aircraft landing may include, but is not limited to, the following: integrating object detections over time by detecting possible new and existing object tracks, assigning object detections to object tracks, and filtering object detections; estimating object state based on the integrated object detections; predicting future object movement and trajectory preview based on the estimated object state, dynamic model, and track history; determining the probability of runway incursion based on the predicted future object movement; and adapting the detection over time to new environments based on the accuracy of the probability.

[0098] This application involves the following terms:

[0099] 1. A method (700) for determining the probability of runway incursion during aircraft landing, the method (700) comprising:

[0100] The object detections are integrated over time (702) by detecting possible new and existing object tracks, assigning object detections to the object tracks, and filtering the object detections;

[0101] (704) object state is estimated based on integrated object detection;

[0102] Based on the estimated object state, dynamic model (303) and track history, predict (706) future object movement and trajectory preview;

[0103] Based on the predicted future object movement, determine the probability of the runway intrusion (708); and

[0104] Based on the accuracy of the probability, the object detection is adapted to the new environment over time (710).

[0105] 2. The method (700) according to Clause 1, wherein detecting possible new and existing object tracks includes:

[0106] Access the track history, sensor measurements (309) associated with the possible new and existing object tracks, and the track preview.

[0107] 3. The method (700) according to Clause 2, further comprising:

[0108] Detecting and tracking non-cooperative objects in a pre-selected area during the aircraft's landing; and

[0109] The probability of the runway incursion is determined based on the following: visual detection probability, number of scans of the preselected area associated with the aircraft landing, number of updates to detected new and existing object tracks, preselected false negative threshold, and preselected false positive threshold.

[0110] 4. The method (700) according to Clause 3, further comprising:

[0111] The object detection is assigned to the non-cooperative object using statistical gating and auction algorithms.

[0112] The statistical gating includes a rectangular gate based on state covariance to filter the object detections considered for allocation.

[0113] The auction algorithm is configured to use the distance between the non-cooperative objects for allocation, and

[0114] Among them, object detections that were not used to maintain existing tracks were assigned to new trial tracks.

[0115] 5. The method (700) according to Clause 3, wherein the preselected area is based on the boundary of the airport runway, the boundary of the taxiway system associated with the airport runway, and the boundary of the runway safety area associated with the airport runway.

[0116] 6. The method (700) according to Clause 5, wherein the airport runway comprises one of the following:

[0117] The defined area for landing and takeoff of the aircraft, the taxiway for ground movement of the aircraft, the explosion-proof mat, the over-limit area, the water surface, the narrow runway for landing training of the aircraft adjacent to the defined area, the vertical takeoff and landing airport and the helicopter airport.

[0118] 7. The method (700) according to Clause 3, wherein the preselected false negative threshold includes a preselected maximum allowed value for false negatives, and wherein the preselected false positive threshold includes a maximum cumulative probability of false positives.

[0119] 8. The method (700) according to Clause 3, further comprising:

[0120] When the ratio of the number of scans to the number of updates reaches a threshold based on the preselected false negative threshold and the preselected false positive threshold, the runway intrusion is declared.

[0121] 9. The method (700) according to Clause 3, further comprising:

[0122] The filtering of non-cooperative object tracks is based on track history, sensor measurements (309), and track previews generated by the dynamic model (303), the filtering including:

[0123] The trajectory preview of the non-cooperative object is estimated by providing the trajectory history to a first Kalman filter (301);

[0124] The acceleration of the non-cooperative object is estimated by providing the trajectory preview and the sensor measurements (309) to a second Kalman filter; and

[0125] The trajectory of the non-cooperative object is smoothed by providing the trajectory history, the sensor measurements (309), and the estimated acceleration to a Gaussian process.

[0126] 10. The method (700) according to Clause 9, further comprising:

[0127] Receive the sensor measurement value (309);

[0128] Estimate the acceleration of the possible new and existing objects;

[0129] The trajectory preview is determined based on the estimated acceleration; and

[0130] Predicted state and covariance estimation of generated object trajectories.

[0131] 11. The method (700) according to Clause 10, wherein estimating (704) the acceleration of the possible new and existing objects includes:

[0132] The trajectory preview was determined using a modified Kalman filter.

[0133] 12. The method (700) according to Clause 11, further comprising:

[0134] Smooth the trajectory of the object; and

[0135] An inference was obtained.

[0136] 13. The method (700) according to Clause 12, wherein smoothing the trajectory of the object comprises:

[0137] The object trajectory, the sensor measurement (309), and the trajectory preview are fed into the Gaussian process.

[0138] 14. The method (700) according to Clause 13, wherein, for the Gaussian process:

[0139] The linear time-varying stochastic differential equation is:

[0140]

[0141] Wherein, state x(t) and input u g (t) is:

[0142] x = [x1, y1, x2, y2] T , ug (t) = [u x ,u y ] T (15),

[0143] Where x1 and y1 are the object's position, and x2 and y2 are the object's velocity.

[0144] The system matrices A(t), B(t), and N(t) are:

[0145]

[0146] The process noise w(t) is:

[0147]

[0148] A stationary zero-mean Gaussian process includes a symmetric positive definite power spectral density matrix Q. c And the Dirac delta function δ, where the solution to equation (14) is:

[0149]

[0150] Where Φ(t,s) is the following transition matrix:

[0151]

[0152] The mean of equation (14) is:

[0153]

[0154] Since the mean is zero, the last term in equation (20) is averaged with respect to w(t) in equation (14);

[0155] The second Kalman filter in equation (8) is used to obtain an estimate of the acceleration input.

[0156] Where, the average value of vector μ is

[0157] in,

[0158]

[0159] The covariance function is based on the following equation:

[0160]

[0161] The covariance components between the two different times are:

[0162]

[0163] Where, for i = 1…M and

[0164]

[0165] The maximum a posteriori (MAP) trajectory is:

[0166]

[0167] The prior P(x) of the Gaussian process is:

[0168]

[0169] Among them, the likelihood P(D(x) i )|x) is:

[0170]

[0171] Calculate the state x and the measurement function g(x) to transform the nonlinear optimization into a linear optimization:

[0172]

[0173] Transform the nonlinear optimization problem in equation (27) using equation (30):

[0174]

[0175] Iteration Until the convergence criterion is satisfied after solving the linear system in equation (31); and

[0176] The linear optimization in equation (31) can be rewritten as:

[0177]

[0178] Among them, Q in equation (26) i and

[0179] 15. The method (700) according to Clause 14, wherein the trajectory preview is generated by the dynamic model (303) comprising:

[0180] x k+1 =A k x k +B k u k y k =C k x k (1); and

[0181] x k =[x1,k ,x 2,k ] T ,

[0182] Where is u k Input; y k It is output, and

[0183] The system matrix is:

[0184]

[0185] 16. The method (700) according to Clause 15, wherein the predicted state and the covariance estimate for each of the object trajectories comprise:

[0186]

[0187] The predicted state is:

[0188]

[0189] Wherein, the system matrices Fk and Hk, and the noise covariance Qk are:

[0190]

[0191] The update process for the predicted state and the updated covariance includes the following innovation and innovation covariance:

[0192]

[0193] Among them, the measurement covariance R k yes

[0194] The optimal Kalman gain is:

[0195]

[0196] The updated predicted state and the updated covariance estimate are:

[0197]

[0198] The fitting residual after measurement is:

[0199] and

[0200] The prediction of the first Kalman filter (301) is:

[0201]

[0202] 17. The method (700) according to Clause 16, further comprising:

[0203] This will accelerate the estimation The preview is fed into the dynamic model (303) to obtain the preview.

[0204] 18. A computer system for determining the probability of runway incursion during aircraft landing, the computer system comprising:

[0205] Hardware processor; and

[0206] A non-volatile storage medium that stores instructions, which, when executed by the hardware processor, perform operations including:

[0207] Detecting and tracking non-cooperative objects in a pre-selected area during the aircraft's landing; and

[0208] The probability of the runway incursion is determined based on the following: visual detection probability, number of scans of the preselected area associated with the aircraft landing, number of updates to detected new and existing object tracks, preselected false negative threshold, and preselected false positive threshold.

[0209] 19. The computer system described in Clause 18 further includes:

[0210] The filtering of non-cooperative object tracks is based on track history, sensor measurements (309), and track previews generated by a dynamic model (303), the filtering including:

[0211] The trajectory preview of the non-cooperative object is estimated by providing the trajectory history to a first Kalman filter (301);

[0212] The acceleration of the non-cooperative object is estimated by providing the trajectory preview and the sensor measurements (309) to a second Kalman filter;

[0213] The trajectory of non-cooperative objects is smoothed by providing the trajectory history, sensor measurements (309), and estimated acceleration to a Gaussian process.

[0214] The object detections are integrated over time (702) by detecting possible new and existing object tracks, assigning object detections to the object tracks, and filtering the object detections;

[0215] (704) object state is estimated based on integrated object detection;

[0216] Predict (706) future object movement and trajectory preview based on the estimated object state, the dynamic model (303), and the track history;

[0217] The probability of runway intrusion (708) is determined based on the predicted future object movement; and

[0218] Based on the accuracy of the probability, the object detection adapts to the new environment over time (710).

[0219] 20. A computer program product for determining the probability of runway incursion during aircraft landing, the computer program product comprising a computer-readable storage medium having program instructions for implementing the computer-readable storage medium, the program instructions being executable by a computing device to cause the computing device to perform operations including:

[0220] Detect and track non-cooperative objects in a pre-selected area during the aircraft's landing;

[0221] The probability of runway incursion is determined based on the following: visual detection probability, number of scans of the pre-selected area associated with the aircraft landing, number of updates to detected new and existing object tracks, pre-selected false negative threshold, and pre-selected false positive threshold.

[0222] The filtering of non-cooperative object tracks is based on track history, sensor measurements (309), and track previews generated by a dynamic model (303), the filtering including:

[0223] The trajectory preview of the non-cooperative object is estimated by providing the trajectory history to a first Kalman filter (301);

[0224] The acceleration of the non-cooperative object is estimated by providing the trajectory preview and the sensor measurements (309) to a second Kalman filter;

[0225] The trajectory of the non-cooperative object is smoothed by providing the trajectory history, the sensor measurements (309), and the estimated acceleration to a Gaussian process;

[0226] The object detections are integrated over time by detecting possible new and existing object tracks, assigning object detections to the object tracks, and filtering the object detections;

[0227] (704) object state is estimated based on integrated object detection;

[0228] Predict (706) future object movement and trajectory preview based on the estimated object state, the dynamic model (303), and the track history;

[0229] The probability of runway intrusion (708) is determined based on the predicted future object movement; and

[0230] Based on the accuracy of the probability, the object detection adapts to the new environment over time (710).

[0231] While the numerical ranges and parameters described in this disclosure are approximate, the numerical values ​​described in the specific embodiments are reported as precisely as possible. However, any numerical value inherently includes some error that must be caused by the standard deviation found in their respective test measurements. Furthermore, all ranges disclosed herein should be understood to encompass any and all subranges contained herein.

[0232] While the invention has been shown with respect to one or more embodiments, changes and / or modifications may be made to the illustrated examples without departing from the spirit and scope of the appended claims. Additionally, while a particular feature of the invention may have been disclosed with respect to only one of several embodiments, such features may be combined with one or more other features of other embodiments, as may be desired and advantageous for any given or particular function. As used herein, the terms “a,” “an,” and “the” may refer to one or more elements or portions of elements. As used herein, the terms “first” and “second” may refer to two distinct elements or portions of elements. As used herein, the term “at least one of A and B” in relation to a list of items (e.g., A and B) means A alone, B alone, or A and B. Those skilled in the art will recognize that these and other variations are possible. Furthermore, with regard to the use of the terms “comprising,” “including,” “having,” “with,” or variations thereof in the detailed description and claims, these terms are intended to be inclusive in a manner similar to the term “comprising.” Furthermore, in the discussion and claims herein, the term “about” indicates that the listed values ​​may be slightly varied, provided that such variation does not cause the process or structure to be inconsistent with the intended purpose described herein. Finally, "exemplary" indicates that the description is used as an example, rather than implying that it is ideal.

[0233] It should be understood that the above-disclosed variations and other features and functions, or alternatives thereof, can be combined into many other different systems or applications. Those skilled in the art can subsequently make various alternatives, modifications, variations, or improvements that are not currently foreseeable or anticipated, and these alternatives, modifications, variations, or improvements are also intended to be covered by the claims.

Claims

1. A method (700) for determining the probability of runway incursion during aircraft landing, the method (700) comprising: The object detections are integrated over time (702) by detecting possible new and existing object tracks, assigning object detections to the object tracks, and filtering the object detections; (704) object state is estimated based on integrated object detection; Based on the estimated object state, dynamic model (303) and track history, predict (706) future object movement and trajectory preview; The probability of runway intrusion (708) is determined based on the predicted future object movement; as well as Based on the accuracy of the probability, the object detection is adapted to the new environment over time (710).

2. The method (700) according to claim 1, wherein, Detecting possible new and existing object tracks includes: Access the track history, sensor measurements (309) associated with the possible new and existing object tracks, and the track preview.

3. The method (700) according to claim 2, further comprising: Detect and track non-cooperative objects in a pre-selected area during the aircraft's landing; as well as The probability of the runway incursion is determined based on the following: visual detection probability, number of scans of the preselected area associated with the aircraft landing, number of updates to detected new and existing object tracks, preselected false negative threshold, and preselected false positive threshold.

4. The method (700) according to claim 3, further comprising: The object detection is assigned to the non-cooperative object using statistical gating and auction algorithms. The statistical gating includes a rectangular gate based on state covariance to filter the object detections considered for allocation. The auction algorithm is configured to use the distance between the non-cooperative objects for allocation, and Among them, object detections that were not used to maintain existing tracks were assigned to new trial tracks.

5. The method (700) according to claim 3, wherein, The preselected area is based on the boundaries of the airport runway, the boundaries of the taxiway system associated with the airport runway, and the boundaries of the runway safety area associated with the airport runway.

6. The method (700) according to claim 5, wherein, The airport runway includes one of the following: The defined area for landing and takeoff of the aircraft, the taxiway for ground movement of the aircraft, the explosion-proof mat, the over-limit area, the water surface, the narrow runway for landing training of the aircraft adjacent to the defined area, the vertical takeoff and landing airport and the helicopter airport.

7. The method (700) according to claim 3, wherein, The preselected false negation threshold includes a preselected maximum allowed value for false negations, and wherein the preselected false affirmation threshold includes the maximum cumulative probability of false affirmations.

8. The method (700) according to claim 3, further comprising: When the ratio of the number of scans to the number of updates reaches a threshold based on the preselected false negative threshold and the preselected false positive threshold, the runway intrusion is declared.

9. The method (700) according to claim 3, further comprising: The filtering of non-cooperative object tracks is based on track history, sensor measurements (309), and track previews generated by the dynamic model (303), the filtering including: The trajectory preview of the non-cooperative object is estimated by providing the trajectory history to a first Kalman filter (301); The acceleration of the non-cooperative object is estimated by providing the trajectory preview and the sensor measurements (309) to a second Kalman filter; and The trajectory of the non-cooperative object is smoothed by providing the trajectory history, the sensor measurements (309), and the estimated acceleration to a Gaussian process.

10. The method (700) according to claim 9, further comprising: Receive the sensor measurement value (309); Estimate the acceleration of the possible new and existing objects; The trajectory preview is determined based on the estimated acceleration; as well as Predicted state and covariance estimation of generated object trajectories.

Images