Method and system for real-time performance evaluation of an air traffic surveillance system, as well as a real-time management method for said surveillance system

Abstract

A method and a system are described for the real-time evaluation of the performance of a cooperative air traffic surveillance system, designed to monitor the presence and movements of mobiles (A, C) within a surveillance region (CV), wherein a plurality of receiving stations (R1-R3; R) is configured to receive respective identification signals from mobiles, wherein the surveillance region (CV) includes at least one operational service volume (SV) and is divided into monitoring cells. A plurality of identification signals associated with a respective mobile (A; C) present in the surveillance region (CV) over a predetermined time interval forms a track or report of the mobile. A plurality of surveillance system performance metrics obtainable from the identification signals is selected, and the values of the selected performance metrics are calculated for at least each monitoring cell that at least partially covers an operational service volume (SV) and for each monitoring cell where at least one selected metric has values below a predetermined minimum value, an overall performance level or system state within the cell is determined according to a predefined correlation matrix among the selected evaluation metrics.

Description

[0001] Method and system for real-time performance evaluation of an air traffic surveillance system, as well as a real-time management method for said surveillance system

[0002] The present invention generally relates to air traffic control and, in particular, to air traffic surveillance systems, more specifically to non-radar air traffic surveillance systems, whether deployed on the ground, in space, or in a hybrid ground-space configuration.

[0003] More specifically, the present invention concerns a method for evaluating the performance of a cooperative air traffic surveillance system according to the preamble of claim 1 , a system for evaluating the performance of a cooperative air traffic surveillance system according to the preamble of claim 7, as well as a real-time management method for said surveillance system.

[0004] Cooperative air traffic surveillance systems, unlike radar surveillance systems that detect the position of an aircraft by analyzing the reflection of interrogation signals emitted by a transmitting system and passively reflected by the aircraft, rely on the transmission of identification signals from specific equipment installed on board aircraft (or airport vehicles), containing data indicative of the identity of the transmitting mobile (also identified as a target), its position, and any other information derived from onboard systems.

[0005] The identification signals are detected by ground or space stations for air traffic surveillance purposes in the vicinity, or by equipment installed on board other aircraft for surrounding air traffic surveillance, for example, to detect the spacing or separation of an aircraft from other aircrafts crossing its path.

[0006] Among the systems that adopt cooperative air traffic surveillance techniques are ADS-B systems and multilateration (MLAT) systems.

[0007] ADS-B (Automatic Dependent Surveillance - Broadcast) systems are a means by which aircrafts or airport vehicles transmit and / or receive identification data, position, speed, and, depending on the case, additional information derived from onboard systems (e.g., GNSS satellite navigation systems and inertial platforms) in a broadcast mode. The system can operate using different digital transmission technologies from an onboard transponder, the most common of which is Mode S, "squitter," operating at a frequency of 1090 MHz. The ADS-B system is automatic because it does not require any external stimulus or interrogation for data transmission, it is dependent on data collected from onboard systems and transmission instruments, and — since the data is broadcast, as opposed to point-to-point transmission between an aircraft and a control centre — the original source does not know the users of its data. Any user, whether an aircraft or a ground station within the coverage region of the broadcast signal, can choose to process the received data for its own purposes without any bilateral service contract.

[0008] The ability of a ground station to receive an ADS-B signal is limited by natural obstacles (terrain elevations) or artificial obstacles (buildings).

[0009] Multilateration systems for air traffic control operate instead by deploying a certain number of receivers and / or receiver / transmitters at predetermined sites within or near an airport. Multilateration is based on the difference in the time of arrival of a signal emitted by a mobile and received by multiple receivers distributed over an area. The time of arrival at each receiver, through the estimation of the speed at which the signal travels through the air, is proportional to the distance travelled by the signal itself, and in this way, it is possible to determine the position of a mobile by solving, for example, the mathematical intersection of multiple hyperbolas (or hyperboloids) constructed on the time of arrival of the same signal at the system receivers distributed in the territory. To obtain a unique two-dimensional measurement, at least three receivers are required, along with a synchronization device for the receivers. In this case, the technique can be defined as trilateration. To obtain a three- dimensional measurement, four receivers and a synchronization device are required. Increasing the number of receivers improves localization accuracy.

[0010] Multilateration systems can be used in passive or active configurations. In a passive configuration, the multilateration system is based on a spontaneous Mode S, "squitter", transmission by aircraft onboard transmitters. In an active configuration, the multilateration system is based on the transmission of asynchronous responses or synchronous responses by aircraft onboard transmitters in response to interrogations from ground stations. The system detects each signal emitted by an aircraft transmitter at different reception sites. The signal is locally time-stamped and then sent to a central processing system, or it is sent directly to the central processing system to be time-stamped centrally. At the central processing system, the difference in the time of arrival of the signal at each receiver is used to estimate the transmitter’s position. Data from three receivers allow the system to determine a two- dimensional position estimate. Data from a fourth receiver are necessary to determine a three-dimensional position estimate. Alternatively, a three-dimensional position estimate can be calculated using only three receivers when altitude can be determined from an external source (such as Mode C code or the "on-ground" bit from the aircraft transmitter).

[0011] The accuracy of the position and the probability of detection of a multilateration system depend on a series of factors, including: the accuracy of time-stamping and the correct decoding of received signals; the position of the receivers relative to the mobile, i.e., the geometric dilution of precision (GDOP); and the number of receivers.

[0012] The efficiency of a multilateration system depends on the ability of each ground station to determine the exact time of arrival and correctly decode the received signals.

[0013] Wide Area Multilateration (WAM) systems are based on the principle of multilateration and are used for surveillance over extensive geographical regions, beyond the limits of the airport premises.

[0014] Airports are generally challenging environments for the propagation of electromagnetic waves, and signal reflection (multipath propagation) continuously occurs due to the presence of numerous objects of significant size relative to the wavelengths of the signals used by the system. Signals resulting from multipath propagation that overlap can affect both the accuracy of time-stamping and the quality of decoding, specifically when the signal superimposed on the original signal has a similar power level and a short delay (e.g., on the order of hundreds of nanoseconds) compared to the original signal. Various signal processing techniques, message processing techniques, and position estimation methods can be employed to minimize the effects of reflected signals.

[0015] The accuracy of position calculation also largely depends on GDOP, meaning that the positions of the receivers (ground stations) may be favourable or unfavourable for a given position of the mobile. An unfavourable position may result in high GDOP values that significantly increase position error, even if accuracy in detecting the time difference of arrival is maintained. A greater angular separation between the receivers used to calculate the target position results in lower GDOP values and improved accuracy in position detection.

[0016] Therefore, the position of receiving stations is crucial for the overall performance of a surveillance system, whether it is an ADS-B, MLAT, or WAM system.

[0017] The number of receivers is determined based on the required coverage volume, which may be an airport or an extensive geographical region. Additional receivers are sometimes needed to overcome the effects of shadowing by buildings, multipath reflections, and to improve the overall positioning accuracy of the surveillance system.

[0018] In general, a cooperative air traffic surveillance system operating with one of the aforementioned techniques has the architecture shown in Figure 1. Aircraft and ground vehicles, for example, aircraft taxiing, taking off, or landing, as well as ground vehicles moving within the airport premises, are denoted as A and C, respectively.

[0019] A network of receivers is represented in the figure, for example, by three ground stations Rl, R2, and R3 (although, in a broader sense, the receivers may also be installed on satellites), each of which is configured to receive a Mode S signal from aircraft and ground vehicles (and, optionally, to transmit an interrogation signal to them), with the received signal conveying a message including position data, identity, and other operational parameters of each mobile. A central processing system, denoted as S, is the node where signals from the receiving stations are directed to determine the set of transmitting mobiles based on the data contained in the received messages. The central processing system S is configured to generate periodic reports on the mobiles to be sent to a data distribution system D.

[0020] External and internal control and monitoring systems, denoted as CMS, are connected to the central processing system S and designed to provide operators with system configuration and control functions, including the provision of reports on the current operational status of the system and its individual components. Generally, CMS control and monitoring systems are configured to transmit information related to the system operating mode and status through a means agreed upon locally with the end user (e.g., an SNMP, HTTP protocol, etc.).

[0021] A surveillance system typically defines three different coverage volumes, as shown in Figure 22.

[0022] The Tracking Coverage Volume denoted as TV is the volume of airspace in which the system is technically capable of tracking aircraft, or more generally, the spatial volume in which the system is able to track mobiles equipped to communicate with an ADS-B or MLAT (WAM) surveillance system.

[0023] The Operational Coverage Volume denoted as CV is the established geographical volume of interest within which the surveillance system is capable of providing surveillance with the required performance. The Operational Coverage Volume may be divided into different Operational Service Volumes denoted as SV, each with its own surveillance performance requirements. A Tracking Coverage Volume extends partially outside the Operational Coverage Volume.

[0024] An Operational Service Volume SV is a subvolume of the Operational Coverage Volume CV in which the surveillance system provides a specific surveillance service. A predetermined Operational Coverage Volume can be divided into multiple Operational Service Volumes, each with its own surveillance performance requirements.

[0025] A surveillance system must meet the agreed-upon performance requirements within the Operational Coverage Volume, hereinafter also generally referred to as the surveillance region. Among the essential performance requirements for the acceptance of an airport site configuration and for any subsequent surveillance service control, such as functional tests (reboot, start & stop) to verify the correct operation of the surveillance system, are, for example, the Probability of Update (PU) and the Probability of Long Gaps (PLG). The Probability of Update, PU, is a known metric that indicates the frequency with which a surveillance system can provide a surveillance report on mobiles present within a service volume, or in other words, the percentage of signals emitted by a mobile that the surveillance system is able to correctly receive. The Probability of Long Gaps, PLG, is a known metric that indicates portions of a trajectory, between two consecutive updates, where position or altitude information of a mobile is missing when the trajectory segment without an update is longer than a predetermined update interval. Such performance requirements are verified on-site through the analysis of surveillance report data obtained from ground stations arranged according to the current configuration.

[0026] In principle, the required performance must be met at every point within the service volume and must be maintained over time.

[0027] The performance requirements apply to mobiles located within the Operational Service Volume(s) SV. Within these volumes, performance requirements must be achieved both globally and for each possible trajectory of the mobile. Furthermore, performance requirements must be evaluated geographically to identify the concentration of low- performance trajectories in specific areas.

[0028] The statistical significance of the PU and PLG metrics depends on the number of samples, i.e., the track data (Target Reports) emitted over time, which collectively form the tracks of the aircraft (and vehicles) monitored by the system receiving stations within an acquisition time interval. The statistical confidence with which performance values can be measured against their minimum requirements therefore depends on the sample size. A smaller sample will provide lower confidence in the measured performance values.

[0029] The performance requirements are defined by EUROCAE standards, specifically by ED-117 standard (Minimum Operational Performance Specification for Mode S Multilateration Systems for use in Advanced Surface Movement Guidance and Control Systems, November 2003), ED-129 (Technical Specification for a 1090 MHz Extended Squitter ADS-B Ground System), and ED- 142 (Technical Specification for Wide Area Multilateration (WAM), September 2010). For example, the Probability of Update is required to be greater than 97% in ADS-B systems, while the Probability of Long Gaps is required to be less than 0.185% in an ATC low-density ATM sector.

[0030] Currently, to ensure compliance with performance requirements, such as those imposed by the above-mentioned national or international standards, a performance evaluation of a surveillance system is conducted once, after system installation, either under real traffic conditions or in a simulated test condition. The indicative performance data of the current system are compared with the required performance data specified by the standards and if the system meets the performance requirements of the standard, the surveillance system, meaning the deployed configuration of ground receiving stations, is accepted and can be made operational.

[0031] Disadvantageously, during the normal operational lifetime of a surveillance system, its performance is no longer evaluated, even though it may deteriorate, meaning it may no longer be guaranteed to comply with the specifications dictated by the aforementioned standards. Performance degradation may occur due to the degradation of one or more receiving stations, the presence of a new obstacle, (either temporary or permanent), within the site, which affects the electromagnetic propagation of signals transmitted by mobiles, for example, due to the construction of a new building or the prolonged stationing of an aircraft, or due to a failure in a system component that is not promptly detected and repaired.

[0032] The present invention thus aims to provide a satisfactory solution to the stated problem, avoiding the drawbacks of the known art. In particular, the present invention aims to verify over time the compliance of a cooperative air surveillance system with the required performance requirements. A further object of the invention is to remedy the deterioration in the performance of a cooperative air surveillance system that was guaranteed at the time of the installation of said system. According to the present invention, this objective is achieved by means of a method having the characteristics set forth in claim 1.

[0033] Specific embodiments are the subject matter of the dependent claims, the content of which is to be understood as an integral or integrating part of the present description.

[0034] A further subject matter of the invention is a system having the characteristics set forth in claim 7.

[0035] A further subject matter of the invention is a real-time management method for a surveillance system, as claimed.

[0036] In summary, the present invention relates to a method and a real-time management system for a cooperative air surveillance system and, in particular, to a method and system for evaluating the performance of a cooperative air surveillance system, and finds application in the provision of a service for the real-time evaluation of the performance of cooperative air surveillance systems such as ADS-B, MLAT, and WAM, (whether terrestrial, space-based, or operating across both terrestrial and space domains), in accordance with national and international standards, such as EUROCAE ED 129, ED 117, and ED 142.

[0037] The real-time evaluation of the performance of an ADS-B, MLAT, or WAM surveillance system is based on the real-time assessment of predetermined performance metrics (for example, PU, PLG) and on verifying the compliance of the receiving stations of the surveillance system with predetermined performance levels, corresponding to the attainment of pre-established minimum values of the selected performance metrics.

[0038] If the predetermined performance levels are not met, a method is proposed for identifying the causes of performance inadequacy and for their resolution.

[0039] Indicative data of the signals detected by the receiving stations of the surveillance system and transmitted by mobiles (aircraft or vehicles) in accordance with the ADS-B, MLAT, and WAM standards are provided in real-time by the surveillance system to an evaluation system according to the invention, hosted on a remote or local server with respect to the surveillance system.

[0040] The evaluation system performs a preliminary filtering of the data to verify its integrity and quality, for example, by checking the validity of Mode S, ensuring that the data sample is not contaminated by data from other monitoring systems, and applying predetermined evaluation algorithms to the selected data to assess the current performance of the surveillance system. The evaluation results are advantageously stored and sent to a user interface, where they are displayed in real-time. Conveniently, the system is also configured to evaluate and display the trends of selected performance metrics over a predetermined period of time.

[0041] If a receiving station provides degraded performance or if a temporary obstacle obstructs the operational service volume, the overall performance of the surveillance system decreases and if the degraded surveillance system does not meet the performance requirements imposed by the standards, the evaluation system diagnoses a performance degradation of the surveillance system and is programmed to issue an alert or alarm notification, for example, via a user interface, indicating a degradation of one or more individual ground receiving stations or an inadequacy in the overall arrangement of the ground receiving stations.

[0042] More specifically, the evaluation system according to the invention executes, at a predefined periodicity (for example, 30 minutes or 1 hour) settable by an operator, predetermined evaluation algorithms to perform periodic checks on a predetermined number of performance metrics of the surveillance system in its installation configuration and is configured to diagnose a degradation in the performance of the surveillance system when the trend over time of the acquired values of a predetermined set of performance metrics of the surveillance system indicates a decline, or a prolonged decline, of such values below a respective predefined acceptability threshold.

[0043] In a currently preferred embodiment, the evaluation system operates based on the correlation of a pair of metrics and, specifically the correlation between the Probability of Update, PU and the Probability of Long Gaps, PLG. Other metrics that can be considered include signal latency or, in the case of multilateration or WAM systems, Horizontal Position Accuracy, HP A, which is defined as the difference between the position reported by a mobile and the reference point position of the mobile at the time of the mobile's report. In the case of multilateration, for HPA evaluation, the reference data used is provided by the ADS-B system (i.e., GPS).

[0044] Other embodiments may be conceived that are based on the correlation of three or more performance metrics.

[0045] It is preferable to select performance metrics that are mutually independent, such as PU and HPA, or weakly correlated, for example, with correlation values lower than 0.5.

[0046] A non-exhaustive list of performance metrics includes PU (Probability of Update), PFTR (Probability of False Target Report), PID (Probability of Identification), PFID (Probability of False Identification), RPA (Reported Position Accuracy), HPA (Horizontal Position Accuracy), PLG (Probability of Long Gaps), Probability of Incorrect, Latency, and Ghost.

[0047] Further characteristics and advantages of the invention will be more clearly explained in the following detailed description of an embodiment, given by way of non-limiting example, with reference to the appended drawings, in which:

[0048] Figure 1 schematically shows a surveillance system according to the known art;

[0049] Figure 2 shows the coverage volumes of a surveillance system according to the known art;

[0050] Figure 3 schematically shows an evaluation system according to the invention associated with a surveillance system;

[0051] Figure 4 represents the subdivision into cells of an operational service volume with an indication of the corresponding performance levels in a condition of compliance with performance requirements;

[0052] Figure 5 is a time representation diagram of a performance metric for two operational service volumes of a surveillance system;

[0053] Figure 6 is another representation of the subdivision into cells of an operational service volume with an indication of the corresponding performance levels in a condition of non-compliance with performance requirements;

[0054] Figures 7a and 7b are correlation matrices between two performance metrics used in the evaluation system according to the invention;

[0055] Figure 8 is a flowchart of an evaluation process of a surveillance system according to the invention.

[0056] With reference to figure 3, the essential features of an evaluation system for a cooperative air surveillance system according to the invention are shown.

[0057] In figure 3, elements or components identical or functionally equivalent to those illustrated in figure 1 have been designated with the same references.

[0058] Number 10 generally denotes a cooperative air traffic surveillance system operating with one or more combined techniques such as ADS-B, MLAT, and WAM, designed to monitor the presence and movements of mobiles, such as aircraft A and ground vehicles C, within a surveillance region, such as for example manoeuvring or taxiing aircraft, taking off or landing aircrafts, and ground vehicles moving within the airport premises, as well as aircraft in flight in the case of surveillance systems with extended geographical coverage.

[0059] Rl, R2, and R3 designate three ground receiving stations arranged within the surveillance region, each configured to receive over time Mode S identification signals from aircraft and ground vehicles (and, optionally, to transmit an interrogation signal to them), the received signals carrying a message containing position data, identity, and other operational parameters of each mobile. S designates a central processing system, to which the signals received from the receiving stations Rl, R2, and R3 are directed in order to determine the set of transmitting mobiles based on the data contained in the messages transmitted by the aircraft and ground vehicles. The central processing system S is configured to generate periodic reports on the mobiles detected within the surveillance region, to be sent to a data distribution system D.

[0060] External and internal control and monitoring systems connected to the central processing system S and configured to provide operators with configuration and control functions for the surveillance system are designated as CMS.

[0061] An evaluation system for the surveillance system according to the invention, located either in proximity to or remote from the surveillance system, is designated as 20 and includes at least a first data processing server SI, which is associated with a database M configured to store the position coordinates of the receiving stations and the terrain map in which these receiving stations are installed. The terrain map includes data representing buildings and, in general, obstacles to the propagation of electromagnetic waves.

[0062] In a preliminary phase, following the installation of the surveillance system, its testing, and validation, the evaluation system 20 is configured to acquire from the central processing system S the values of selected performance metrics required by the standards, for example, but not limited to, the values of PU and PLG.

[0063] When the surveillance system 10 is in operational mode, the central processing system S transmits to the data processing server SI of the evaluation system the track data of the mobiles currently acquired over a time interval, for example, through a known transmission protocol such as ASTERIX CAT 10 / 20 / 21 / 48 / 62 or a forthcoming protocol (ASTERIX Cat 25).

[0064] The data processing server SI of the evaluation system is configured to calculate the number of samples at a given moment or during an acquisition time interval. Based on the number of samples, the evaluation outcome may be of a first or second type.

[0065] A first type of evaluation outcome, referring exemplarily to the PU metric, is shown in Table 1, where the index ka is correlated to the number of samples by the formula where Nui is the number of samples.

[0066] In this case, up to seven different performance levels of the surveillance system are defined (column 2), depending on the determined update probability value (column 1).

[0067] A second type of evaluation outcome, referring exemplarily to the PU metric, is shown in Table 2.

[0068] In this case, where the number of samples available for determining the Probability of Update is limited, a lower number of performance levels of the surveillance system is defined (column 2), depending on the determined Probability of Update value (column 1). According to a conservative approach, the evaluation system is restricted in providing a positive assessment outcome.

[0069] In general, the values of the performance metrics are determined over a configurable time interval based on a set of signals for each track of a mobile, as well as for each operational volume and for each monitoring cell into which a surveillance region is subdivided. For predefined ranges of metric values, corresponding performance levels are defined. The number of performance levels, and in particular the number of acceptable performance levels (four out of seven in the first type of evaluation outcome, one out of four in the second type of evaluation outcome), is defined as a function of the number of available samples. If a performance level is below a predetermined performance threshold, it is marked as unacceptable.

[0070] In the example provided in the previous tables, the acceptable performance levels are those marked as VERY HIGH PASS, HIGH PASS, MEDIUM PASS, NOMINAL PASS, and POSSIBLE PASS. The unacceptable performance levels are those marked as MEDIUM FAIL, HIGH FAIL, and VERY HIGH FAIL.

[0071] The selected performance metrics are evaluated in real time for each track of a mobile and for each Operational Service Volume, SV, or Operational Coverage Volume, CV, for example, by acquiring series of data as shown in the following table:

[0072] Within each Operational Service Volume, SV, or Operational Coverage Volume, CV, the aforementioned selected performance metrics are evaluated per tracks, per the overall volume, and per "cells," as represented in Figure 4. Figure 4 shows an Operational Coverage Volume CV within which three different Operational Service Volumes SV are identified, for example, corresponding to runways, taxiing areas, or airport terminals. The receiving ground stations are identified as R. The Operational Coverage Volume CV is subdivided into cells, for example, polygonal cells, preferably square cells and in the figure, the relevant cells are shown in dark tones, overlaid on the three identified Operational Service Volumes SV. Along the margins of the coverage volume representation, two performance level scales are shown, corresponding respectively to the first and second type of evaluation outcome, depending on the number of available samples (track measurements). The relevant cells, shown in dark tones, are marked with a performance level corresponding to the POSSIBLE PASS level of the second type of evaluation outcome.

[0073] The data is stored in a database DB associated with the SI server, to retain the historical record of measurements and the trends of performance metric values over time.

[0074] Visualization means DY may be provided to visually show to an operator the values of the performance metrics of the tracks of mobiles acquired in real time, per Operational Service Volumes SV and per cells.

[0075] Figure 5 illustrates, as an example, a graph representing the trend over time (historical record) of the Probability of Update, PU, as a selected performance metric. For reference only, the expected PU metric values REQ PU and the measured values MEAS PU are shown, for a first (REQ PU l, MEAS PU l) and a second (REQ PU 2, MEAS PU 2) Operational Service Volume.

[0076] If the selected performance metrics do not meet the reference standards, the evaluation system is configured to issue a warning or an alarm. To prevent false alarms and ensure a more robust evaluation system, a data processing logic is applied in a secondary server S2, which is connected to the SI server to receive the measured performance metric data. This logic will be explained further below.

[0077] The performance metrics of all tracks are evaluated, and tracks that do not meet the reference standards are identified.

[0078] Based on the set of selected metric values, the system removes a track, that is, the set of metric values related to signals emitted by a mobile, if the minimum required value for at least one selected metric is not met, this is done to clean the set of selected metric values from the effects of signals not acquired by the receiving stations of the surveillance system due to transmission defects, for example, due to anomalies in an aircraft onboard transmitting system, by an aircraft not configured to transmit signals, or in case of insufficient signal power (due to distance or electromagnetic wave reflection by obstacles), and therefore not attributable to a malfunction of one or more ground receiving stations or an invalid arrangement of the ground receiving stations in the surveillance system.

[0079] Specifically, the evaluation system is configured to remove a track that does not meet the reference values of the selected metric when it is the only track failing to meet those reference values.

[0080] If a single track does not meet the reference values of the selected metric, for example, based on the data shown in the following table the track (in this case, track 4cad3b) is excluded from the overall evaluation because it may originate from aircraft for which the service is not provided (for example, due to insufficient transmission power or a special qualification of the aircraft) or from aircraft whose onboard avionics exhibit functional anomalies. In the latter case, it is preferable that the aircraft be evaluated separately to confirm the anomaly, for instance, through a cross-check of the onboard ADS-B system with a cooperative data report before being excluded from the evaluation.

[0081] In such cases, no alarm is issued.

[0082] Conversely, if it is determined that a plurality of tracks has performance metric values below a respective predetermined threshold corresponding to the reference values of the selected metric, the evaluation system performs a volume analysis by subdividing the surveillance region into cells and evaluating, for each cell, an overall performance level or system state according to a predefined correlation matrix between the selected evaluation metrics.

[0083] Figure 6 shows the Operational Coverage Volume CV in Figure 4, within which the same Operational Service Volumes SV identified in Figure 4 are represented. The receiving ground stations are identified as R. The Operational Coverage Volume CV (which here corresponds to a surveillance region) is subdivided into cells, for example, polygonal cells, preferably square cells and in the figure, the relevant cells are shown in grayscale tones, overlaid on the three identified Operational Service Volumes SV. Along the margin of the coverage volume representation, the performance level scales are displayed for both the first and second type of evaluation outcome, depending on the number of available samples (track measurements). The relevant cells include a first set of cells, shown in darker tones, marked with a performance level corresponding to the VERY HIGH PASS level of the first type of evaluation outcome, a second set of cells, shown in lighter tones, is marked with a performance level corresponding to the POSSIBLE PASS level of the second type of evaluation outcome, and a third set of cells, shown in intermediate tones, is marked with a performance level corresponding to the VERY HIGH FAIL level of the second type of evaluation outcome.

[0084] In this case, the server S2 performs an analysis based on the cells, using the values of at least two selected performance metrics, for example, PU and PLG. The cells in which the reference values of at least one performance metric are not met are identified, as shown in Figure 6 and based on the cells that do not meet the required performance standards, the evaluation system is configured to conduct the following investigation. The system assesses the number of available measurement samples ("Grade ka Value") for at least two selected metrics, in this case PU and PLG, and an overall performance level or system state is determined according to a predefined correlation matrix between the selected evaluation metrics, as shown, for example, in Figures 7a and 7b, respectively for the first and second types of evaluation outcomes, depending on the number of available samples. The system state is indicated at the intersection of the rows and columns of the matrix for each combination of performance levels of the two selected metrics. A system state marked as "ok" is considered acceptable, while states of the system marked as "warning" or "alarm" are considered unacceptable. If the evaluation system (through the server S2) determines that at least one cell within an Operational Service Volume SV, meaning a cell entirely or partially contained in an Operational Service Volume SV, has an overall performance level below a first predetermined performance threshold (which depends on the combination of the performance levels of the individual metrics) and is marked as unacceptable, the evaluation system assigns an "alert" status to the cell, this status can be signalled to an operator through the DY visualization means and if the alert status persists for a period longer than a predefined time threshold (for example, two hours), the evaluation system determines an "alarm" state, which can also be signalled to an operator through the DY visualization means. If the evaluation system (through the S2 server) determines that at least one cell within an Operational Service Volume SV i.e. a cell that is totally or partially included in Operational Service Volume SV has an overall performance level below a second predetermined performance threshold (which depends on the combination of the performance levels of the individual metrics) and is marked as unacceptable, the evaluation system assigns an "alarm" status to the cell, which can be signalled to an operator through the DY visualization means.

[0085] When an alarm state is detected, the evaluation system 20 is also configured to support the identification of the issue that caused the alarm state.

[0086] In the event of an alarm state, the following operations may be conducted: (1) Identification of the ground receiving station exhibiting anomalies; (2) Identification of obstacles on the terrain map wherein the surveillance system is applied.

[0087] In the first case, the server S2 of the evaluation system searches for the presence of a receiving station within the cell identified as having an unacceptable performance level. If no receiving station is detected, the server S2 identifies the receiving station that is spatially closest to the cell with an unacceptable performance level. At the identified receiving station, potential failures in the station's connection components of the receiving station at the system network, at the receiving station software and hardware are checked, for example, through a connection and status request by the server S2 to the central processing system S of the surveillance system 10. In the event of a failure, the surveillance system will be temporarily reconfigured, switching the faulty receiving station to maintenance mode, meaning it is removed from the surveillance system, and the surveillance system performance is re-evaluated. If a plurality of cells with unacceptable performance levels is identified and multiple receiving station failures are confirmed, the surveillance system will enter an "alarm" mode.

[0088] Conversely, if after inspection or automated software verification no failures are detected in the receiving stations, the evaluation system deduces that the performance degradation of the surveillance system is due to the presence of an obstacle along the propagation path of the signals, which could be either a temporary or permanent obstacle.

[0089] In this second case, the evaluation system proceeds with further checks on the surveillance system conditions, specifically, the system identifies and marks the receiving station within the cell that does not meet the required performance standards, or if no receiving stations are present within that cell, it marks the nearest receiving station to the cell that does not meet the preformed standards. A new terrain map is then acquired, and a comparison is performed between the new map and the previously stored map in the database M, which relates to the previous surveillance system installation period. If new permanent obstacles, indicated as B in Figure 3, are identified from the comparison, an automated simulation of a predefined planning process is conducted to determine a better location for the receiving station by a planning server S3, using the current map, which includes the identified obstacle(s) B, to determine an optimized position for the identified receiving station or an optimized configuration of the receiving stations within the surveillance system and reconfigure the surveillance system. The new receiving station position is evaluated considering the "shadow cone" effect caused by the new obstacle B on the propagation of electromagnetic signals.

[0090] Following the reconfiguration or partial reconfiguration of the surveillance system using a data-driven approach, the PU and PLG metric values of the surveillance system evaluation system as a whole are simulated by the planning server S3 to verify compliance with performance objects. If the simulated PU and PLG metrics meet the specified reference standards, the reconfigured receiving station or stations can be made operational. An evaluation method for a surveillance system according to the invention, implemented by an evaluation system as described above, can be executed by running one or more computer programs stored locally at each of the SI, S2, and S3 servers or remotely, with execution being, for example, distributed across the SI, S2, and S3 servers.

[0091] This method is exemplified in Figure 8 and includes the following steps.

[0092] At step 100, a map of the territory in which the surveillance system is deployed is prepared, containing position information for the respective stations and at step 110, the surveillance system is made operational. At the following step 120, the values of selected performance metrics are determined based on the plurality of identification signals received from mobiles at the stations over a predetermined time interval, for acquired tracks, operational volumes, and subdivision cells of the surveillance region.

[0093] At step 140, it is verified for each track whether at least one performance metric has values lower than a corresponding predetermined threshold. If at least one performance metric for a single track has values below the corresponding predetermined threshold, that track is excluded from the data to be analysed for evaluating the performance of the surveillance system. If multiple tracks have one or more performance metrics with values below their respective thresholds, at step 160, the level of at least two selected performance metrics is evaluated for each cell, based on which, at step 180, a system state is determined. If the system state is unacceptable (indicated as "warning" or "alarm"), the process advances to step 200, where the causes of performance inadequacy are identified. At steps 220 and 240, in the case of an alarm state, a search for a malfunctioning ground receiving station and an identification of obstacles on the territory map wherein the surveillance system is applied are conducted, respectively. In the first case, still at step 220, the surveillance system is temporarily reconfigured by removing the identified receiving station from the surveillance system, and the surveillance system performance is re-evaluated starting from step 110. If the performance degradation is due to the presence of an obstacle along the signal propagation path, still at step 240, permanent obstacles are identified, an improved arrangement of the affected receiving stations is calculated, the map is updated at step 100, and the performance of the surveillance system is re-evaluated starting from step 110.

[0094] It should be noted that the proposed implementation of the present invention described above is purely exemplary and not limiting to the present invention. A skilled person in the field could easily implement the present invention in different embodiments that do not deviate from the principles outlined herein and are therefore included within the scope of this patent.

[0095] This applies particularly to the possibility of using different performance metrics, for example, for MLAT and WAM surveillance systems, the Horizontal Position Accuracy, HPA.

[0096] Of course, without departing from the principle of the invention, the embodiments and implementation details may be widely modified compared to what has been described and illustrated purely as a non-limiting example, without departing from the scope of protection of the invention as defined in the appended claims.

Claims

CLAIMS1. A method for evaluating, in real time, the performance of a cooperative air traffic surveillance system for monitoring the presence and movements of mobiles (A, C) within a surveillance region (CV), said surveillance system comprising a plurality of receiving stations (R1-R3; R) arranged within said surveillance region (CV) and adapted to receive from said mobiles (A, C) respective identification signals, wherein said surveillance region (CV) comprises at least one operational service volume (SV) and is divided into monitoring cells, the method comprising the steps of acquiring identifying signals associated with a set of mobiles (A, C) present in said surveillance region (CV) at a pre-determined time interval, a plurality of identifying signals associated with a respective mobile forming a track or report of said mobile selecting a plurality of surveillance system performance metrics obtainable from said identification signals with which there are associated predetermined ranges of values, adapted to define a plurality of surveillance system performance levels, and a predetermined performance threshold, wherein performance levels above said predetermined threshold are marked as acceptable and performance levels below said predetermined threshold are marked as unacceptable; calculating values of said selected performance metrics from said plurality of identification signals received in a pre-defined time interval, for each track or report of a mobile; if a track or report of a mobile shows values of at least one selected metric below a predetermined minimum value for said selected metric, excluding the values of the identification signals of said track or report from said acquired signals thus forming a collection of residual identification signals; on the basis of said collection of residual identification signals, calculating values of said selected performance metrics for at least each monitoring cell at least partially covering an operational service volume (SV), and, for each monitoring cell having values of at least one selected metric lower than a predetermined minimum value for said selected metric, determining an overall performance level or system state in said cell according to a predetermined correlation matrix between said selected evaluation metrics,wherein if said overall performance level is below a first predetermined performance threshold, said cell is marked with an alert status; if said overall performance level is below a second predetermined performance threshold lower than said first predetermined performance threshold, said cell is marked with an alarm status.

2. Method according to claim 1, wherein if the alert state associated with a cell whose overall performance level is below said first predetermined performance threshold persists for a time interval greater than a predetermined time threshold, said cell is marked with an alert state.

3. Method according to any one of the preceding claims, wherein said selected performance metrics include at least two of PU (Probability of Update), PFTR (Probability of False Target Report), PID (Probability of Identification), PFID (Probability of False Identification), RPA (Reported Position Accuracy), HPA (Horizontal Position Accuracy), PLG (Probability of Long Gaps), Probability of Incorrect, Latency, Ghost.

4. Method according to claim 3, wherein said selected performance metrics include mutually independent or weakly correlated performance metrics.

5. Method according to claim 4, wherein said selected performance metrics include Probability of Upgrading, PU, and Probability of Long Gaps, PLG.

6. Method according to any one of the preceding claims, wherein from said acquired identifying signals are excluded values of identifying signals of a track or report having values of at least one selected metric that are lower than a predetermined minimum value for said selected metric when said track is the only track that has values of at least one selected metric lower than the predetermined minimum value for said metric.

7. A system for real-time performance evaluation of a cooperative air traffic surveillance system for monitoring the presence and movements of mobiles (A, C) within asurveillance region (CV), said surveillance system comprising a plurality of receiving stations (R1-R3; R) arranged within said surveillance region (CV) and adapted to receive respective identification signals from said mobiles (A, C), wherein said surveillance region (CV) comprises at least one operational service volume (SV) and is divided into monitoring cells, the system comprising data processing means (S, SI, S2) programmed to: acquiring identification signals associated with a set of mobiles (A, C) present in said monitoring region (CV) at a pre-determined time interval, a plurality of identification signals associated with a respective mobile forming a track or report of said mobile, selecting a plurality of surveillance system performance metrics obtainable from said identification signals with which there are associated predetermined ranges of values, adapted to define a plurality of surveillance system performance levels, and a predetermined performance threshold, wherein performance levels above said predetermined threshold are marked as acceptable and performance levels below said predetermined threshold are marked as unacceptable; calculating values of said selected performance metrics from said plurality of identification signals received in a pre-defined time interval, for each track or report of a mobile; if a track or report of a mobile shows values of at least one selected metric below a predetermined minimum value for said selected metric, excluding the values of the identification signals of said track or report from said acquired signals thus forming a collection of residual identification signals; on the basis of said collection of residual identification signals, calculating values of said selected performance metrics for at least each monitoring cell at least partially covering an operational service volume (SV), and, for each monitoring cell having values of at least one selected metric lower than a predetermined minimum value for said selected metric, determining an overall performance level or system state in said cell according to a predetermined correlation matrix between said selected evaluation metrics, wherein if said overall performance level is below a first predetermined performance threshold, said cell is marked with an alert status; if said overall performance level is below a second predetermined performancethreshold lower than said first predetermined performance threshold, said cell is marked with an alarm status.

8. Real-time managing method for a surveillance system, comprising the steps of storing the position coordinates of the receiving stations (R1-R3; R) in the surveillance region (CV) and data representative of obstacles to the propagation of said identifying signals in the surveillance region (CV) at least at a previous installation time implementing a method for real-time performance evaluation of an air traffic surveillance system according to any one of claims 1 to 6, for each cell marked with an alarm status identifying the problem that caused the alarm status, wherein identifying the problem that caused the alarm status includes identifying at least one receiving station (R) with anomalies or identifying obstacles in the surveillance region (CV).

9. Method according to claim 8, wherein identifying at least one receiving station (R) having anomalies includes: identifying a receiving station (R) within the cell marked with said alarm status, or, if no re-receiving station (R) is present within said cell, identifying a receiving station (R) spatially closer to the cell marked with said alarm status, verifying the presence of a fault at said receiving station (R) and, if a fault exists, reconfiguring the surveillance system by switching said receiving station (R) to maintenance mode and removing said receiving station (R) from the surveillance system; and implementing said method for real time performance evaluation of an air traffic surveillance system according to any one of claims 1 to 6, on the reconfigured surveillance system.

10. Method according to claim 8, wherein identifying obstacles in the surveillance region comprises: identifying a receiving station (R) within the cell marked with said alarm status, or, if no receiving station (R) is present within said cell, identifying a receiving station (R) spatially closer to the cell marked with said alarm status, acquiring up-to-date data representative of obstacles to the propagation of saididentifying signals in the surveillance region (CV) and comparing said up-to-date data with stored data representative of obstacles at an earlier time of installation to identify new obstacles in the surveillance region (CV), if new obstacles are identified, carrying out a simulation of a pre-determined planning process of the surveillance system in the surveillance region (CV) thus determining a different layout of at least said receiving station (R).

Images