A method for automated system-level verification of DAA warning and guidance

Abstract

The application relates to the technical field of software testing, in particular to an automatic system-level verification method for DAA (Data-Link Autonomous Attitude) alarm and guidance. According to a verification path specified by a current test case, initial flight path data corresponding to the DAA system under test is input; according to a special degradation scene corresponding to the test case, to-be-calculated flight path data is output; an alarm and guidance algorithm module is called to execute preset DAA alarm logic operation, actual alarm level data is generated, and the actual alarm level data is output as an actual alarm result file in a preset format; and according to the actual alarm result file, a preset reference alarm result file and a scene description file, analysis and multi-level compliance determination are carried out to generate a determination conclusion of the current test case. The automatic verification is realized from case data injection, flight path data preprocessing, alarm calculation to result determination.

Description

Technical Field

[0001] This invention relates to the field of software testing technology, and specifically to an automated system-level verification method for DAA alarms and guidance. Background Technology

[0002] With the widespread integration of unmanned aerial vehicles (UAS) into national airspace systems, Detect and Avoidance (DAA) systems have become core components for ensuring air traffic safety and preventing mid-air collisions. RTCA DO-365C, as the industry's minimum operational performance standard (MOPS) for UAV DAA systems, imposes stringent requirements on the system's alarm logic, guidance mechanisms, and testing and verification. In the engineering implementation of the RTCA DO-365C standard, the entire software engineering data flow typically manifests as follows: First, raw trajectory data from multiple sources such as radar, ADS-B, and electro-optical sensors is received. After filtering and correlation calculation by the front-end trajectory processing and fusion module, the flight status data of the UAV and the center trajectory data of the intruding aircraft are output. Subsequently, the center trajectory data is input in real-time to the alarm and guidance core algorithm module, which calculates and outputs the final alarm level and guidance avoidance zone according to the standard requirements. To ensure the correctness, stability, and standard compliance of the engineering implementation, a complete software engineering verification and testing methodology must be constructed to conduct comprehensive, embedded-deployment-oriented closed-loop verification of standard compliance.

[0003] In existing technologies, current verification is mostly limited to complete, ideal data input, failing to systematically cover the various special cases specified in the DO-365C standard. These include situations such as: tracking non-cooperative targets using only air-to-air radar (ATAR) or electro-optical / infrared (EO / IR) sensors; missing azimuth or altitude information of intruding aircraft; and ADS-B tracks that have not been validated. In these data degradation scenarios, the standard requires algorithms to suppress preventative alarms, exempt early warnings from time requirements, or expand the boundaries for determining no-hazard zones. However, existing alarm and guidance algorithms have not yet been adapted to meet the standard requirements for these specific degradation scenarios, resulting in gaps in compliance verification for data degradation scenarios. Summary of the Invention

[0004] The purpose of this invention is to provide an automated system-level verification method for DAA alarms and guidance, which solves the problems in the prior art.

[0005] This invention is achieved through the following technical solution:

[0006] In a first aspect, embodiments of the present invention provide an automated system-level verification method for DAA alarms and guidance, comprising:

[0007] According to the verification path specified by the current test case, the corresponding initial trajectory data is input into the DAA system under test; the verification path includes a first path and a second path. The first path is a full-link system-level verification path, used to verify the system-level functional performance of the integrated trajectory processing module and alarm and guidance algorithm module; the second path is an independent verification path for alarm and guidance algorithms, used to independently verify the functional and standard compliance of the alarm and guidance algorithm module.

[0008] According to the preset test case configuration in the test case, when the test case is configured to require verification of special degradation scenarios, the corresponding degradation status flag is written into the initial trajectory data; when the test case is configured to be a standard normal scenario, the initial trajectory data is kept unchanged, and the trajectory data to be calculated is output.

[0009] Based on the trajectory data to be calculated, the alarm and guidance algorithm module is invoked to perform a preset DAA alarm logic operation, generating actual alarm level data corresponding to each moment, and outputting the generated actual alarm level data as an actual alarm result file in a preset format;

[0010] The system analyzes and performs multi-level compliance checks based on the actual alarm result file, the preset reference alarm result file, and the scenario description file to generate the judgment conclusion for the current test case.

[0011] Preferably, the step of inputting the corresponding initial trajectory data into the DAA system under test according to the verification path specified by the current test case includes:

[0012] When the verification path is the first path, preset multi-source sensor raw data is obtained, and the multi-source sensor raw data is input to the track processing module. After the track processing module performs filtering and correlation calculation, the first track data is output and used as the initial track data. The multi-source sensor raw data comes from the test case set provided by the RTCA DO-365C standard, and includes one or more of radar data, ADS-B data, and photoelectric data.

[0013] When the verification path is the second path, the preset true trajectory data is obtained, and the true trajectory data is used as the initial trajectory data; the true trajectory data is ideal simulation data without noise and sensor error, including the true trajectory data of the local machine and the true trajectory data of the intruder.

[0014] Preferably, before using the true track data as the initial track data, the method further includes:

[0015] Based on a unified time tag, the local true trajectory data and the intrusion machine true trajectory data are merged to generate a trajectory data file that conforms to the input interface specification of the alarm and guidance algorithm module;

[0016] Use the aforementioned track data file as the initial track data.

[0017] Preferably, when the initial track data is the first track data output by the track processing module under the first path, the method further includes:

[0018] The system acquires LOG format data from the raw data of the multi-source sensors and TRACK data output after processing by the track processing module. The LOG format data is the raw data recording format specified in the RTCA DO-365C standard, and the TRACK data is the fused track data output by the track processing module after filtering and correlation calculation of the LOG format data.

[0019] Using the same data source as the matching criterion, compare the deviation between the TRACK data and the LOG format data;

[0020] Based on whether the deviation falls within the preset track processing tolerance range, the processing effect verification conclusion of the track processing module is output.

[0021] Preferably, writing the corresponding degraded status flag into the initial track data includes:

[0022] When the special downgrade scenario is a scenario where the altitude information of the intruding aircraft is missing, an altitude missing flag is written into the initial flight track data;

[0023] When the special downgrade scenario is a scenario where the azimuth information of the intruding aircraft is missing, an azimuth missing flag is written into the initial track data;

[0024] When the special degradation scenario is a scenario that relies solely on photoelectric or infrared sensors to track non-cooperative targets, a non-cooperative flag is written into the initial track data;

[0025] When the special downgrade scenario is a scenario that relies solely on air-to-air radar to track non-cooperative targets, a radar-only flag is written into the initial track data;

[0026] When the special downgrade scenario is that the ADS-B track data has not been validated, an unvalidated ADS-B flag is written into the initial track data.

[0027] Preferably, the alarm and guidance algorithm module includes detection logic, alarm logic, and maneuver guidance logic; the step of calling the alarm and guidance algorithm module to perform a preset DAA alarm logic operation based on the trajectory data to be calculated includes:

[0028] Based on the trajectory data to be calculated, the relative motion situation parameters between the local machine and the intruding machine are calculated through the detection logic. The relative motion situation parameters include relative distance, relative speed, heading cross angle, minimum interval distance, and minimum interval time.

[0029] And based on the relative motion situation parameters, predict the time interval during which the security interval between the local machine and the intrusion machine will be breached;

[0030] Based on the time interval and the preset danger alarm zone parameters and non-danger zone parameters, the alarm logic outputs the actual alarm level data for each moment.

[0031] Based on the relative motion situation parameters and the actual alarm level data at each moment, the maneuver guidance logic generates guidance and avoidance zone data corresponding to the actual alarm level data at each moment.

[0032] The actual alarm level data and guidance / avoidance data at each moment are output in the form of a structured file to obtain the actual alarm result file.

[0033] Preferably, the parsing based on the actual alarm result file, the preset reference alarm result file, and the scenario description file includes:

[0034] Based on the actual alarm result file, the actual alarm level at each moment is parsed and extracted to obtain the actual alarm level sequence;

[0035] Based on the reference alarm result file, the minimum alarm level and maximum alarm level at each moment are parsed and extracted to obtain the reference alarm level interval sequence;

[0036] Based on the scenario description document, the alarm exemption rules for non-cooperative target blind zone, alarm judgment tolerance rules, alarm delay grace rules, and early alarm tolerance judgment rules are parsed and extracted.

[0037] Preferably, the multi-level compliance determination includes:

[0038] Based on the actual alarm level at any specified time in the actual alarm level sequence, and the minimum and maximum alarm levels corresponding to the specified time in the reference alarm level interval sequence, determine whether the actual alarm level falls within the closed interval formed by the minimum alarm level and the maximum alarm level. If so, output a pass determination conclusion.

[0039] If the actual alarm level does not fall within the closed interval, the actual alarm level is further evaluated according to the alarm delay grace rule, the alarm judgment tolerance rule, the early alarm tolerance judgment rule, and the non-cooperative target blind zone alarm exemption rule. If the actual alarm level meets the conditions specified by any one of the rules, the alarm delay grace rule, the early alarm tolerance judgment rule, and the non-cooperative target blind zone alarm exemption rule, a pass judgment conclusion is output; otherwise, a failure judgment conclusion is output and the failure type is marked as missed, false alarm, or delayed alarm according to the deviation direction.

[0040] Preferably, the method further includes:

[0041] Based on the judgment conclusion of the current test case, a test report for the current test case is generated. The test report includes at least the test case identifier, input data summary, actual alarm level data, reference output results in the reference alarm result file, judgment conclusion, and failure type. The failure type is determined by the multi-level compliance judgment based on the deviation direction when outputting the failure judgment conclusion.

[0042] Based on the judgment conclusion, update the iteration status identifier of the current test case in the global test sequence table. The iteration status identifier includes at least the test-pending status, the passed status, and the repair-needed status. When the judgment conclusion is passed, update the iteration status identifier to the passed status. When the judgment conclusion is failed, update the iteration status identifier to the repair-needed status, and record the failure type in the global test sequence table.

[0043] Preferably, the method further includes:

[0044] Based on the iteration status identifier of each test case in the global test sequence table, determine whether there are test cases marked as needing repair in the global test sequence table;

[0045] The testing process terminates when no test cases marked as requiring repair exist in the global test sequence list.

[0046] When there are test cases marked as needing repair in the global test sequence list and the preset iteration termination condition has not been met, the regression test process is triggered in response to the repair and update operation of the DAA system under test corresponding to the needing repair state.

[0047] The regression testing process is as follows: based on the repair status identifiers in the global test sequence list, the test cases corresponding to the repair status are selected, the selected test cases are re-executed with automated verification steps, and the test is terminated when the iteration status identifiers of all test cases in the global test sequence list are in the passed state or the preset iteration termination condition is met.

[0048] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0049] Through a dual-path data injection mechanism, the track processing module and the alarm and guidance algorithm module form a continuous data stream during system-level verification. This avoids the interruption of the execution chain caused by the asynchronous input of front-end output data to the back-end after being transferred through a static file, ensuring that end-to-end dynamic characteristics are reflected in the integrated verification. Before the track data is input to the alarm and guidance algorithm module, various sensor data constraints are simulated by writing downgraded status flags. This allows the special scenarios specified in the standard to be executed within the same verification framework without the need to build a separate dedicated test environment, ensuring that the algorithm adaptation requirements for the special scenarios specified in the standard are systematically covered in system-level verification. The alarm and guidance algorithm module receives the tagged track data, performs alarm logic operations, and outputs a formatted actual alarm result file, providing a unified input source for subsequent judgment. Through the joint parsing of the actual alarm result file, the reference alarm result file, and the scenario description file, as well as the execution of the two-level judgment logic, the compliance judgment of the alarm level is automatically completed without manual intervention. This achieves automated execution and closed-loop feedback of the complete verification chain from use case data injection, track data preprocessing, alarm calculation to result judgment. Attached Figure Description

[0050] To more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort. In the drawings:

[0051] Figure 1 This is a flowchart illustrating the automated system-level verification method for DAA alarms and guidance provided by the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0053] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0054] It should be noted that all actions involving the acquisition of signals, information, or data in this invention are carried out in compliance with the relevant data protection laws and regulations of the locality and with authorization from the owner of the relevant device.

[0055] Example 1

[0056] Please see Figure 1 This invention provides an automated system-level verification method for DAA alarms and guidance, comprising:

[0057] S1. Input the corresponding initial trajectory data into the DAA system under test according to the verification path specified by the current test case; the verification path includes a first path and a second path. The first path is a full-link system-level verification path, used to verify the system-level functional performance of the integrated trajectory processing module and alarm and guidance algorithm module; the second path is an independent verification path for alarm and guidance algorithms, used to independently verify the functional and standard compliance of the alarm and guidance algorithm module.

[0058] Each test case refers to a single verification instance designed for the alarm and guidance functions of the DAA system. Each test case includes an input data source identifier, test case configuration, and expected output reference data. Each test case has a unique identifier.

[0059] The DAA system under test refers to a complete perception and avoidance software system undergoing verification method testing. The DAA system under test must contain at least two functional units: a trajectory processing module and an alarm and guidance algorithm module. The DAA system under test is deployed and runs in an embedded target environment or a simulation test environment. The DAA system under test receives externally injected trajectory data as input and outputs alarm level data and guidance and avoidance strip data.

[0060] The verification path refers to the data flow channel followed when test data is injected into the system under test. The verification path includes two possible values: the first path and the second path.

[0061] The first path refers to the complete data channel from the input of raw data from multiple sensors to the trajectory processing module, the fusion output of the trajectory processing module, and then the input to the alarm and guidance algorithm module. The first path is used to verify the overall functional performance of the integrated trajectory processing module and alarm and guidance algorithm module.

[0062] The second path refers to the data channel through which the merged and processed standard true track data is directly input to the alarm and guidance algorithm module. This second path is used to perform functional verification and standard compliance verification of the alarm and guidance algorithm module independently.

[0063] Initial track data refers to the input data injected into the DAA system under test under the verification path specified in the current test case. Under the first path, the initial track data is the fused track data output after the raw data from multiple sensors has been fused by the track processing module. Under the second path, the initial track data is the track data file formed after merging and processing standard ground truth track data.

[0064] The track processing module is a functional unit that performs filtering and multi-sensor data correlation calculations on raw data from multiple sensors. It receives radar data, ADS-B data, and photoelectric data as input and outputs fused track data.

[0065] The alarm and guidance algorithm module refers to the set of algorithms that are independently tested under the second path. The alarm and guidance algorithm module includes at least detection logic, alarm logic, and maneuver guidance logic. The alarm and guidance algorithm module does not include a track processing module.

[0066] System-level functional performance refers to the degree to which the overall output alarm level and guidance band, driven by a continuous data stream, conforms to standard reference values ​​after the track processing module and alarm and guidance algorithm module are integrated. System-level functional performance reflects the interface compatibility and cumulative error between modules under integrated conditions.

[0067] Standard compliance verification refers to the process of comparing the output of the alarm and bootstrap algorithm module with the reference output provided by the RTCA DO-365C standard. Based on the comparison results, standard compliance verification determines whether the algorithm under test meets the performance requirements specified in the standard.

[0068] Specifically, during implementation, all test cases are pre-configured in tabular form, with each test case associated with a specified verification path value and test case configuration. When the verification path of the current test case is the first path, the raw data from multiple sensors is sent to the track processing module. The track processing module performs coordinate transformation, noise filtering, and track association on the raw data from multiple sensors, and outputs fused track data, which serves as the initial track data for subsequent steps. When the verification path of the current test case is the second path, the truth track data file, which has been merged by time tag, is directly used as the initial track data for subsequent steps. Because two data injection channels are set up, interface compatibility issues between the track processing module and the alarm and guidance algorithm module can be exposed during system-level integration verification. During independent algorithm verification, noise and error interference introduced by the pre-processing stage can be eliminated, thereby evaluating the overall system performance and the algorithm's own performance under integrated conditions. Furthermore, the track processing module and the alarm and guidance algorithm module are components of the DAA software system under test, not new components introduced by this verification method. This verification method is applicable to DAA systems that already have the above-mentioned functional module divisions. The functional definitions and interface specifications of the above modules are determined by the software architecture design of the DAA system. This verification method sets up two verification paths, respectively taking the complete DAA system (including the track processing module and the alarm guidance module) or the alarm and guidance algorithm module (excluding the track processing module) as the test object and integrating them into the test process, thereby achieving verification coverage of different system levels.

[0069] In some implementations, based on the verification path specified by the current test case, the corresponding initial trajectory data is input to the DAA system under test, including:

[0070] When the verification path is the first path, preset multi-source sensor raw data is obtained, and the multi-source sensor raw data is input to the track processing module. After the track processing module performs filtering and correlation calculation, the first track data is output and used as the initial track data. The multi-source sensor raw data comes from the test case set provided by the RTCA DO-365C standard, and includes one or more of radar data, ADS-B data, and photoelectric data.

[0071] When the verification path is the second path, the preset true trajectory data is obtained, and the true trajectory data is used as the initial trajectory data; the true trajectory data is ideal simulation data without noise and sensor error, including the true trajectory data of the local machine and the true trajectory data of the intruder.

[0072] Among them, multi-source sensor raw data refers to the raw measurement data directly from various airborne sensors without fusion processing. Multi-source sensor raw data includes target position information, velocity information, and target identification information independently output by each sensor. The raw data of each type of sensor is recorded using an independent protocol format.

[0073] The RTCA DO-365C standard refers to the minimum operational performance standard document for unmanned aerial vehicle (UAV) DAA systems published by the Radio Technical Committee for Aeronautics (RTCA). The RTCA DO-365C standard specifies the alarm logic, bootstrap generation rules, and test case set for standard compliance verification of the DAA system. The test case set provided by the RTCA DO-365C standard includes raw data files from multiple sensor sources and reference alarm result files.

[0074] Radar data refers to target detection information output by airborne air-to-air radar sensors. Radar data includes range measurements, azimuth measurements, and velocity measurements of the intruding aircraft. Radar data contains inherent measurement errors in both the range and angular dimensions.

[0075] ADS-B data refers to the status information reported by intruders via an Automatic Dependent Surveillance-Broadcast (ADS-B) link. ADS-B data includes the intruder's latitude and longitude location, altitude, velocity vector, and target identification code. ADS-B data may contain false target information or incorrect location reports if its validity has not been verified.

[0076] Optical and photoelectric data refers to target image tracking information output by airborne optical or infrared sensors. This data includes measurements of the intrusion aircraft's line-of-sight angle and angular velocity relative to the host aircraft. However, optical and photoelectric data cannot directly provide distance information or high-precision speed information for the intrusion aircraft.

[0077] True-value trajectory data refers to the ideal target trajectory data generated through simulation. True-value trajectory data is generated by the simulation platform according to a preset conflict scenario. True-value trajectory data does not contain non-ideal factors such as sensor measurement noise, data loss, or false targets.

[0078] Noise-free and sensor error-free ideal simulation data refers to the target's true state vector directly extracted from the simulation environment. In this data, the position coordinates, velocity vector, and time label are all theoretical values ​​set for the simulation. This data does not undergo any sensor measurement equations or filtering processes.

[0079] The true flight path data refers to the sequence of the actual position and velocity vectors of an aircraft equipped with a DAA system at each moment in the simulation scenario. The true flight path data includes the aircraft's longitude coordinates, latitude coordinates, altitude, northbound velocity, eastbound velocity, and vertical velocity.

[0080] Intruder aircraft ground truth track data refers to the true position and velocity vector sequence of the aircraft acting as the conflict target in the simulation scenario at each moment. The intruder aircraft ground truth track data includes the intruder aircraft's longitude coordinates, latitude coordinates, altitude, northbound velocity, eastbound velocity, and vertical velocity. The intruder aircraft ground truth track data is used to merge with the local ground truth track data to generate relative motion situation input.

[0081] Filtering and correlation analysis refers to the computational process of performing noise filtering, coordinate alignment, and correlation matching of multi-source measurement data for the same target on raw data from multiple sensors. Filtering and correlation analysis performs coordinate transformation and time synchronization on radar data, ADS-B data, and electro-optical data respectively. It then determines the correlation of measurement data from different sensors based on target identification codes and spatial positions. Finally, it fuses multiple sets of correlated measurement values ​​for the same target into a single track state estimate using a pre-defined filtering algorithm.

[0082] First track data refers to the fused track data output by the track processing module after filtering and correlation calculation. First track data includes target ID, timestamp, position coordinates, velocity vector, and track quality assessment value. First track data is a single state estimation sequence obtained by fusing multi-source sensor measurement data for the same target. The data format and field definitions of first track data are compatible with the input interface specifications of the alarm and guidance algorithm module.

[0083] Specifically, when performing verification for the first path, the original data files of the multi-source sensors associated with the current test case are pre-read from the test case set provided by the RTCA DO-365C standard. Radar data, ADS-B data, and photoelectric data are decoded according to their respective protocol formats and sent to the track processing module. The track processing module performs time registration on the measurement data from different sensors at the same time, performs noise filtering on the measurement values ​​of each sensor, associates the multi-source measurements belonging to the same intruder based on the target identification code, calculates the track state estimate and its covariance matrix for each group of associated measurement values ​​using a weighted fusion method, and outputs the first track data in the order of sampling time. The first track data is used as the initial track data for subsequent processing steps. When performing verification for the second path, the local ground truth track data and the intrusion machine's ground truth track data are read from the simulation database in advance. Based on a unified time stamp, the local ground truth track data and the intrusion machine's ground truth track data are merged to generate a track data file that conforms to the input interface specification of the alarm and guidance algorithm module. This track data file is then used as the initial track data in subsequent processing steps. Because the first and second paths use track data from different sources and at different processing stages as input, data isolation is achieved between system-level verification and independent algorithm verification. This allows for the exposure of fusion bias introduced during integration and judgment bias inherent in the algorithm itself.

[0084] In some embodiments, when the initial track data is the first track data output by the track processing module under the first path, the method further includes:

[0085] The system acquires LOG format data from the raw data of the multi-source sensors and TRACK data output after processing by the track processing module. The LOG format data is the raw data recording format specified in the RTCA DO-365C standard, and the TRACK data is the fused track data output by the track processing module after filtering and correlation calculation of the LOG format data.

[0086] Using the same data source as the matching criterion, compare the deviation between the TRACK data and the LOG format data;

[0087] Based on whether the deviation falls within the preset track processing tolerance range, the processing effect verification conclusion of the track processing module is output.

[0088] In this context, LOG format data refers to the raw sensor measurement data recording format specified in the RTCA DO-365C standard. LOG format data is organized according to the field order and encoding rules defined in the standard, and includes raw measurement values ​​such as sensor type identifier, measurement timestamp, target distance, target azimuth, and target height. LOG format data is stored as files in the test case set, with each type of sensor corresponding to an independent LOG format data file.

[0089] The raw data record format specified in the RTCA DO-365C standard refers to the encoding specifications for raw measurement data defined in the test case set. This format specifies the field order, data type, unit system, and timestamp encoding method for each type of sensor raw data. This format ensures that sensor data from different sources has a unified parsing rule when input into the system under test.

[0090] TRACK data refers to the fused track record data output by the track processing module after filtering and correlating LOG format data. TRACK data is organized according to the track data structure defined internally by the DAA system under test, and includes a unique target identifier, fused position coordinates, fused velocity vector, track start time, track update time, and track quality assessment value. The correspondence between TRACK data and LOG format data is matched through the identifier field of the same data source.

[0091] The deviation refers to the difference between the estimated target state in the TRACK data and the original measurement value in the LOG format data from the same data source at the same time. The deviation includes three dimensions: position deviation, velocity deviation, and time deviation. Position deviation is the spatial distance difference between the fused position coordinates in the TRACK data and the corresponding original position measurement value in the LOG format data. Velocity deviation is the vector difference between the fused velocity vector and the original velocity measurement value. The deviation is calculated point-by-point according to the preset sampling time points.

[0092] The preset track processing tolerance range refers to the range of maximum allowable deviation values ​​pre-defined for each type of deviation. The preset track processing tolerance range is determined by the DAA system design specifications or set according to the accuracy requirements of the front-end processing stage in the RTCA DO-365C standard. The preset track processing tolerance range includes upper limits for position tolerance, speed tolerance, and time synchronization tolerance.

[0093] Specifically, during the execution of the first path, the trajectory processing module outputs the first trajectory data to the alarm and guidance algorithm module while simultaneously recording intermediate state data during its processing at fixed sampling intervals. This intermediate state data includes the raw measurement values ​​contained in the received LOG format data and the fused estimated values ​​in the TRACK data formed after filtering and correlation calculation. For the multi-source sensor raw data file corresponding to the current test case, the LOG format data records of each sensor are extracted and sorted by timestamp to form the raw measurement sequence. The TRACK data records output by the trajectory processing module are simultaneously extracted and sorted by timestamp to form the fusion estimation sequence. The raw measurement sequence and the fusion estimation sequence are aligned based on the same target ID and timestamp, and the position deviation and velocity deviation at each alignment point are calculated. The position deviation of each sampling point is compared item by item with the preset position tolerance upper limit, and the velocity deviation is compared item by item with the preset velocity tolerance upper limit. If all deviations do not exceed the corresponding tolerance upper limit, the processing effect verification conclusion is marked as passed, indicating that the fusion accuracy of the trajectory processing module meets the input data quality requirements for subsequent alarm calculations. If the deviation exceeds the tolerance limit, the processing effect verification conclusion is marked as failing, and the time and dimension of the deviation are recorded. This information is used to diagnose whether the filter parameter configuration or correlation logic in the track processing module is correct. Through the above comparison process, the fusion quality of the track processing stage is separated from the system-level verification and evaluated independently, making it possible to distinguish the source of front-end processing deviation from back-end alarm judgment deviation.

[0094] S2. According to the preset test case configuration in the test case, when the test case is configured to require verification of special degradation scenarios, write the corresponding degradation status flag into the initial trajectory data; when the test case is configured to be a standard normal scenario, keep the initial trajectory data unchanged and output the trajectory data to be calculated.

[0095] Among them, test case configuration refers to the attribute fields attached to the test cases that are pre-defined by the user, which are used to indicate the type of sensor data input conditions simulated by the current test case.

[0096] Special downgrade scenarios include missing altitude information of the intruding aircraft, missing azimuth information of the intruding aircraft, relying solely on electro-optical or infrared sensors to track non-cooperative targets, relying solely on air-to-air radar to track non-cooperative targets, and ADS-B track data that has not been validated.

[0097] Degradation status flags refer to the status fields written into the initial track data. Degradation status flags are used to indicate the limited status of a certain dimension of information in the track data. Degradation status flags include altitude missing flag, bearing missing flag, non-cooperative flag, radar-only flag, and unverified ADS-B flag. Each type of degradation status flag corresponds to a specific sensor degradation scenario.

[0098] Standard routine scenarios refer to test conditions where sensor data is complete and the target type is known. In standard routine scenarios, the initial trajectory data includes position and velocity information across all dimensions.

[0099] The track data to be calculated refers to the track data output to the alarm and guidance algorithm module after the data quality status marking is completed. The track data to be calculated is based on the initial track data, and whether or not a degradation status flag is added according to the test case configuration.

[0100] Specifically, during implementation, the test case configuration of the current test case is read before the initial track data is input to the alarm and guidance algorithm module. When the test case configuration points to a special degradation scenario verification, the corresponding degradation status flag is written to the protocol field of the initial track data. When the test case configuration points to a standard normal scenario, all fields of the initial track data remain unchanged. After completing the flag writing or holding operation, the processed data is identified as the track data to be calculated and transmitted to the alarm and guidance algorithm module. Because the degradation conditions are simulated and the data is marked before the algorithm calculation, the alarm and guidance algorithm module can automatically adjust the alarm judgment boundary and time window parameters according to the flag, so that the verification range covers the non-ideal sensor input conditions specified in the standard.

[0101] In some implementations, a corresponding degrade status flag is written into the initial track data, including:

[0102] When the special downgrade scenario is a scenario where the altitude information of the intruding aircraft is missing, an altitude missing flag is written into the initial flight track data;

[0103] When the special downgrade scenario is a scenario where the azimuth information of the intruding aircraft is missing, an azimuth missing flag is written into the initial track data;

[0104] When the special degradation scenario is a scenario that relies solely on photoelectric or infrared sensors to track non-cooperative targets, a non-cooperative flag is written into the initial track data;

[0105] When the special downgrade scenario is a scenario that relies solely on air-to-air radar to track non-cooperative targets, a radar-only flag is written into the initial track data;

[0106] When the special downgrade scenario is that the ADS-B track data has not been validated, an unvalidated ADS-B flag is written into the initial track data.

[0107] S3. Based on the trajectory data to be calculated, call the alarm and guidance algorithm module to perform the preset DAA alarm logic operation, generate the actual alarm level data corresponding to each moment, and output the generated actual alarm level data as an actual alarm result file according to the preset format;

[0108] The preset DAA alarm logic operation refers to the process of calculating the relative motion status between the local machine and the intrusion machine and determining the alarm level according to the calculation method specified in the RTCA DO-365C standard. The preset DAA alarm logic operation includes at least the prediction of the safe interval breach time interval and the comparison of parameters between the dangerous alarm zone and the non-dangerous zone.

[0109] Actual alarm level data refers to the alarm level value output by the alarm and guidance algorithm module at each moment. Actual alarm level data reflects the collision risk level between the local machine and the intruding machine under the current relative motion situation.

[0110] The default format refers to the order and encoding rules of the data fields in the output file. The default format must include at least a timestamp field and a numerical field for the alarm level at the corresponding time.

[0111] The actual alarm result file refers to a structured file that records the actual alarm level data at all sampling times according to a preset format. The actual alarm result file is used as input for subsequent judgment steps.

[0112] Specifically, during implementation, the trajectory data to be calculated is sent to the alarm and guidance algorithm module. This module performs relative motion calculations on the state vectors of each local and intruding aircraft, calculating the relative distance, relative speed, heading crossover angle, minimum clearance distance, and minimum clearance time. Based on the calculation results, it predicts the time interval within which the safety interval will be breached. This time interval is compared with the parameters for dangerous and non-dangerous alarm zones to determine the alarm level at each moment. The alarm level values ​​for all sampled moments are written sequentially into a structured file with a preset format, resulting in the actual alarm result file. Because the output file format is consistent with the standard reference file format, subsequent judgment steps can directly perform field-level comparisons.

[0113] In some implementations, based on the trajectory data to be calculated, the alarm and guidance algorithm module is invoked to perform preset DAA alarm logic operations, including:

[0114] Based on the trajectory data to be calculated, the relative motion situation parameters between the local machine and the intruding machine are calculated through the detection logic. The relative motion situation parameters include relative distance, relative speed, heading cross angle, minimum interval distance, and minimum interval time.

[0115] And based on the relative motion situation parameters, predict the time interval during which the security interval between the local machine and the intrusion machine will be breached;

[0116] Based on the time interval and the preset danger alarm zone parameters and non-danger zone parameters, the alarm logic outputs the actual alarm level data for each moment.

[0117] Based on the relative motion situation parameters and the actual alarm level data at each moment, the maneuver guidance logic generates guidance and avoidance zone data corresponding to the actual alarm level data at each moment.

[0118] The actual alarm level data and guidance / avoidance data at each moment are output in the form of a structured file to obtain the actual alarm result file.

[0119] S4. Based on the actual alarm result file, the preset reference alarm result file, and the scenario description file, perform parsing and multi-level compliance determination to generate the determination conclusion of the current test case.

[0120] The preset reference alarm result file refers to the alarm level reference value file provided in the RTCA DO-365C standard for each test case. The preset reference alarm result file records the minimum and maximum alarm levels at each moment.

[0121] The scenario description file refers to the rule description file associated with test cases provided in the RTCA DO-365C standard. The scenario description file contains rules for exemption from alarms in non-cooperative target blind zones, rules for alarm judgment tolerance, rules for alarm delay grace periods, and rules for early alarm tolerance judgment.

[0122] The decision result refers to the compliance verification result output for the current test case. The decision result is either pass or fail. When the decision result is fail, a failure type flag is attached, which includes missed detection, false positive, and late detection.

[0123] Specifically, during implementation, the actual alarm result file, reference alarm result file, and scenario description file are read simultaneously. The actual alarm level value for each moment is extracted from the actual alarm result file; the minimum and maximum alarm levels for each moment are extracted from the reference alarm result file; and the non-cooperative target blind zone alarm exemption rules, alarm judgment tolerance rules, alarm delay grace rules, and early alarm tolerance judgment rules are extracted from the scenario description file. For each moment, it is first determined whether the actual alarm level falls within the closed interval formed by the minimum and maximum alarm levels. If it does, it is considered a pass. If not, the alarm delay grace rules are used to determine whether the time deviation of the actual alarm level is within the allowable delay range; the early alarm tolerance judgment rules are used to determine whether the early alarm output is within the acceptable range; and the non-cooperative target blind zone alarm exemption rules are used to determine whether the current scenario meets the alarm exemption conditions. If any of the conditions specified by the supplementary rules are met, it is considered a pass; otherwise, it is considered a failure, and the failure type is marked according to the deviation direction of the actual alarm level relative to the reference interval. By employing a three-level supplementary rule to review each output that fails the first level, it is possible to distinguish between genuine algorithm defects and deviations allowed by the standard, thus avoiding misjudging standard-allowed outputs as failures.

[0124] In some implementations, the parsing is performed based on the actual alarm result file, the preset reference alarm result file, and the scenario description file, including:

[0125] Based on the actual alarm result file, the actual alarm level at each moment is parsed and extracted to obtain the actual alarm level sequence;

[0126] Based on the reference alarm result file, the minimum alarm level and maximum alarm level at each moment are parsed and extracted to obtain the reference alarm level interval sequence;

[0127] Based on the scenario description document, the alarm exemption rules for non-cooperative target blind zone, alarm judgment tolerance rules, alarm delay grace rules, and early alarm tolerance judgment rules are parsed and extracted.

[0128] During the parsing process, the parser simultaneously loads the actual alarm result file, the reference alarm result file, and the scenario description file. The parser defines and identifies field boundaries and data types according to the respective formats of the three files. The parser reads the timestamp field and alarm level value field line by line from the actual alarm result file, extracting the alarm level value for each sampling time in line number order, and arranging all extracted results chronologically to form the actual alarm level sequence. The parser reads the timestamp field, minimum alarm level field, and maximum alarm level field line by line from the reference alarm result file, extracting the minimum and maximum alarm levels for each sampling time in line number order, and arranging all extracted results chronologically to form the reference alarm level interval sequence. The parser reads each rule entry in the scenario description file, identifying the non-cooperative target blind zone alarm exemption rule, alarm judgment tolerance rule, alarm delay grace period rule, and early alarm tolerance judgment rule based on the rule type identifier, and storing these four types of rules as independent rule objects. After parsing, the actual alarm level sequence, the reference alarm level interval sequence, and the four types of rule objects are all in memory and ready for subsequent judgment logic to call. Since the output data from the three sources has been unified into a sequence structure under the same sampling time base during the parsing stage, the subsequent judgment logic can directly align the actual alarm level and the reference alarm level interval at the same time coordinate, avoiding comparison errors caused by inconsistent data formats.

[0129] In some implementations, the multi-level compliance determination includes:

[0130] Based on the actual alarm level at any specified time in the actual alarm level sequence, and the minimum and maximum alarm levels corresponding to the specified time in the reference alarm level interval sequence, determine whether the actual alarm level falls within the closed interval formed by the minimum alarm level and the maximum alarm level. If so, output a pass determination conclusion.

[0131] If the actual alarm level does not fall within the closed interval, the actual alarm level is further evaluated according to the alarm delay grace rule, the alarm judgment tolerance rule, the early alarm tolerance judgment rule, and the non-cooperative target blind zone alarm exemption rule. If the actual alarm level meets the conditions specified by any one of the rules, the alarm delay grace rule, the early alarm tolerance judgment rule, and the non-cooperative target blind zone alarm exemption rule, a pass judgment conclusion is output; otherwise, a failure judgment conclusion is output and the failure type is marked as missed, false alarm, or delayed alarm according to the deviation direction.

[0132] In some embodiments, the method further includes:

[0133] Based on the judgment conclusion of the current test case, a test report for the current test case is generated. The test report includes at least the test case identifier, input data summary, actual alarm level data, reference output results in the reference alarm result file, judgment conclusion, and failure type. The failure type is determined by the multi-level compliance judgment based on the deviation direction when outputting the failure judgment conclusion.

[0134] Based on the judgment conclusion, update the iteration status identifier of the current test case in the global test sequence table. The iteration status identifier includes at least the test-pending status, the passed status, and the repair-needed status. When the judgment conclusion is passed, update the iteration status identifier to the passed status. When the judgment conclusion is failed, update the iteration status identifier to the repair-needed status, and record the failure type in the global test sequence table.

[0135] In some embodiments, the method further includes:

[0136] Based on the iteration status identifier of each test case in the global test sequence table, determine whether there are test cases marked as needing repair in the global test sequence table;

[0137] The testing process terminates when there are no test cases marked as requiring repair in the global test sequence list.

[0138] When there are test cases marked as needing repair in the global test sequence list and the preset iteration termination condition has not been met, the regression test process is triggered in response to the repair and update operation of the DAA system under test corresponding to the needing repair state.

[0139] The regression testing process is as follows: based on the repair status identifiers in the global test sequence list, the test cases corresponding to the repair status are selected, the selected test cases are re-executed with automated verification steps, and the test is terminated when the iteration status identifiers of all test cases in the global test sequence list are in the passed state or the preset iteration termination condition is met.

[0140] Specifically, during the test initiation phase, a global test sequence table is initialized, and the iteration status identifiers of all test cases are set to the "to be tested" state. Test cases in the "to be tested" states are extracted sequentially according to the order in the global test sequence table and sent to the verification process. After each test case is evaluated, its iteration status identifier in the global test sequence table is updated based on the evaluation result: a "passed" status is written, and a "repair needed" status is written. After all test cases in the "to be tested" states have been executed, the test executor scans the iteration status identifier field of all records in the global test sequence table. If no record marked as "repair needed" is found in the scan results, the test executor determines that all test cases have passed verification and terminates the test process. If a record marked as "repair needed" is found in the scan results, the test executor reads the preset iteration termination condition and compares it with the current iteration round count. If the current iteration round has reached the upper limit specified by the preset iteration termination condition, the test executor forcibly terminates the test process and marks the remaining test cases in the "repair needed" state as timed out and failed. If the current iteration round has not reached the preset iteration termination condition, the test executor enters a waiting state and suspends the triggering of regression tests. After developers complete the analysis and localization of failure issues corresponding to the states requiring repair, and implement code modifications or parameter adjustments to the DAA system under test, a version update operation is performed. Upon detecting a change in the version identifier of the DAA system under test, the test executor filters the corresponding set of test case identifiers based on the state requiring repair identifiers in the global test sequence list, and re-enters the filtered test cases into the verification process to execute all steps. After regression execution is complete, the pass / fail status of the test case is reassessed, and the iteration status identifier is updated. This process is repeated until the iteration status identifiers of all test cases are in a pass state or the preset iteration termination condition is met. Because regression testing only filters test cases for states requiring repair rather than executing all cases, passed test cases are not executed repeatedly, reducing the verification time required for each iteration. Due to the set iteration termination condition, the test process will not fall into an infinite loop when faced with unresolved failure cases.

[0141] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0142] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An automated system-level verification method for DAA alarms and guidance, characterized in that, include: According to the verification path specified by the current test case, the corresponding initial trajectory data is input into the DAA system under test; the verification path includes a first path and a second path. The first path is a full-link system-level verification path, used to verify the system-level functional performance of the integrated trajectory processing module and alarm and guidance algorithm module; the second path is an independent verification path for alarm and guidance algorithms, used to independently verify the functional and standard compliance of the alarm and guidance algorithm module. According to the preset test case configuration in the test case, when the test case is configured to require verification of special degradation scenarios, the corresponding degradation status flag is written into the initial trajectory data; when the test case is configured to be a standard normal scenario, the initial trajectory data is kept unchanged, and the trajectory data to be calculated is output. Based on the trajectory data to be calculated, the alarm and guidance algorithm module is invoked to perform a preset DAA alarm logic operation, generating actual alarm level data corresponding to each moment, and outputting the generated actual alarm level data as an actual alarm result file in a preset format; The system analyzes and performs multi-level compliance checks based on the actual alarm result file, the preset reference alarm result file, and the scenario description file to generate the judgment conclusion for the current test case.

2. The method according to claim 1, characterized in that, The step of inputting the corresponding initial trajectory data into the DAA system under test according to the verification path specified in the current test case includes: When the verification path is the first path, preset multi-source sensor raw data is obtained, and the multi-source sensor raw data is input to the track processing module. After the track processing module performs filtering and correlation calculation, the first track data is output and used as the initial track data. The multi-source sensor raw data comes from the test case set provided by the RTCA DO-365C standard, and includes one or more of radar data, ADS-B data, and photoelectric data. When the verification path is the second path, the preset true trajectory data is obtained, and the true trajectory data is used as the initial trajectory data; the true trajectory data is ideal simulation data without noise and sensor error, including the true trajectory data of the local machine and the true trajectory data of the intruder.

3. The method according to claim 2, characterized in that, Before using the true track data as the initial track data, the method further includes: Based on a unified time tag, the local true trajectory data and the intrusion machine true trajectory data are merged to generate a trajectory data file that conforms to the input interface specification of the alarm and guidance algorithm module; Use the aforementioned track data file as the initial track data.

4. The method according to claim 2, characterized in that, When the initial track data is the first track data output by the track processing module under the first path, the method further includes: The system acquires LOG format data from the raw data of the multi-source sensors and TRACK data output after processing by the track processing module. The LOG format data is the raw data recording format specified in the RTCA DO-365C standard, and the TRACK data is the fused track data output by the track processing module after filtering and correlation calculation of the LOG format data. Using the same data source as the matching criterion, compare the deviation between the TRACK data and the LOG format data; Based on whether the deviation falls within the preset track processing tolerance range, the processing effect verification conclusion of the track processing module is output.

5. The method according to claim 1, characterized in that, The step of writing the corresponding degraded status flag into the initial track data includes: When the special downgrade scenario is a scenario where the altitude information of the intruding aircraft is missing, an altitude missing flag is written into the initial flight track data; When the special downgrade scenario is a scenario where the azimuth information of the intruding aircraft is missing, an azimuth missing flag is written into the initial track data; When the special degradation scenario is a scenario that relies solely on photoelectric or infrared sensors to track non-cooperative targets, a non-cooperative flag is written into the initial track data; When the special downgrade scenario is a scenario that relies solely on air-to-air radar to track non-cooperative targets, a radar-only flag is written into the initial track data; When the special downgrade scenario is that the ADS-B track data has not been validated, an unvalidated ADS-B flag is written into the initial track data.

6. The method according to claim 1, characterized in that, The alarm and guidance algorithm module includes detection logic, alarm logic, and maneuver guidance logic; the step of calling the alarm and guidance algorithm module to execute preset DAA alarm logic operations based on the trajectory data to be calculated includes: Based on the trajectory data to be calculated, the relative motion situation parameters between the local machine and the intruding machine are calculated through the detection logic. The relative motion situation parameters include relative distance, relative speed, heading cross angle, minimum interval distance, and minimum interval time. And based on the relative motion situation parameters, predict the time interval during which the security interval between the local machine and the intrusion machine will be breached; Based on the time interval and the preset danger alarm zone parameters and non-danger zone parameters, the alarm logic outputs the actual alarm level data for each moment. Based on the relative motion situation parameters and the actual alarm level data at each moment, the maneuver guidance logic generates guidance and avoidance zone data corresponding to the actual alarm level data at each moment. The actual alarm level data and guidance / avoidance data at each moment are output in the form of a structured file to obtain the actual alarm result file.

7. The method according to claim 1, characterized in that, The parsing based on the actual alarm result file, the preset reference alarm result file, and the scenario description file includes: Based on the actual alarm result file, the actual alarm level at each moment is parsed and extracted to obtain the actual alarm level sequence; Based on the reference alarm result file, the minimum alarm level and maximum alarm level at each moment are parsed and extracted to obtain the reference alarm level interval sequence; Based on the scenario description document, the alarm exemption rules for non-cooperative target blind zone, alarm judgment tolerance rules, alarm delay grace rules, and early alarm tolerance judgment rules are parsed and extracted.

8. The method according to claim 7, characterized in that, The multi-level compliance determination includes: Based on the actual alarm level at any specified time in the actual alarm level sequence, and the minimum and maximum alarm levels corresponding to the specified time in the reference alarm level interval sequence, determine whether the actual alarm level falls within the closed interval formed by the minimum alarm level and the maximum alarm level. If so, output a pass determination conclusion. If the actual alarm level does not fall within the closed interval, the actual alarm level is further evaluated according to the alarm delay grace rule, the alarm judgment tolerance rule, the early alarm tolerance judgment rule, and the non-cooperative target blind zone alarm exemption rule. If the actual alarm level meets the conditions specified by any one of the rules, the alarm delay grace rule, the early alarm tolerance judgment rule, and the non-cooperative target blind zone alarm exemption rule, a pass judgment conclusion is output; otherwise, a failure judgment conclusion is output and the failure type is marked as missed, false alarm, or delayed alarm according to the deviation direction.

9. The method according to claim 1, characterized in that, The method further includes: Based on the judgment conclusion of the current test case, a test report for the current test case is generated. The test report includes at least the test case identifier, input data summary, actual alarm level data, reference output results in the reference alarm result file, judgment conclusion, and failure type. The failure type is determined by the multi-level compliance judgment based on the deviation direction when outputting the failure judgment conclusion. Based on the judgment conclusion, update the iteration status identifier of the current test case in the global test sequence table. The iteration status identifier includes at least the test-pending status, the passed status, and the repair-needed status. When the judgment conclusion is passed, update the iteration status identifier to the passed status. When the judgment conclusion is failed, update the iteration status identifier to the repair-needed status, and record the failure type in the global test sequence table.

10. The method according to claim 9, characterized in that, The method further includes: Based on the iteration status identifier of each test case in the global test sequence table, determine whether there are test cases marked as needing repair in the global test sequence table; The testing process terminates when there are no test cases marked as requiring repair in the global test sequence list. When there are test cases marked as needing repair in the global test sequence list and the preset iteration termination condition has not been met, the regression test process is triggered in response to the repair and update operation of the DAA system under test corresponding to the needing repair state. The regression testing process is as follows: based on the repair status identifiers in the global test sequence list, the test cases corresponding to the repair status are selected, the selected test cases are re-executed with automated verification steps, and the test is terminated when the iteration status identifiers of all test cases in the global test sequence list are in the passed state or the preset iteration termination condition is met.

Images