Methods for testing aircraft systems, aircraft, and computer-readable storage devices

Abstract

This application relates to a method for testing an aircraft system, an aircraft, and a computer-readable storage device. The method includes: obtaining a first test matrix for a first aircraft system and a second test matrix for a second aircraft system. The method further includes: during a first run test of the first test matrix, obtaining sensor data including second sensor data not specified by the first test matrix. The method includes: evaluating a second run test of the second test matrix by processing the second sensor data using a second analytical model of the second aircraft system. The method further includes: generating second predicted sensor data based on the evaluation of the second run test. The method includes: generating a second error measure by comparing a second subset of the sensor data with the second predicted sensor data. The method includes: determining a test coverage metric of the second test matrix based at least in part on the range of the second sensor data.

Description

[0001] Cross-reference of related applications

[0002] This application claims priority to and is a continuation to pending U.S. Patent Application No. 63 / 064,201, filed August 11, 2020, entitled “AIRCRAFT SYSTEM OPERATIONAL TESTING,” the contents of which are incorporated herein by reference in their entirety. Technical Field

[0003] This disclosure generally relates to the operational testing of aircraft systems. Background Technology

[0004] Operational testing of aircraft systems is typically performed on individual aircraft systems. As a result, a large number of tests must be conducted on many aircraft systems included in a typical aircraft. Performing a large number of tests can be costly and time-consuming. Furthermore, if a change is expected to affect a particular system, operational testing is performed on that specific system, while unintended effects of that change on other aircraft systems may go undetected. Summary of the Invention

[0005] In a particular implementation, a testing method is provided, comprising the steps of: obtaining, at a device, a first test matrix for a first aircraft system and a second test matrix for a second aircraft system. The method further comprises the steps of: during a first run test of the first test matrix, obtaining sensor data including first sensor data within a range specified by the first test matrix and second sensor data not specified by the first test matrix. The method further comprises the steps of: at the device, evaluating the first run test by processing the first sensor data using a first analysis model of the first aircraft system. The method further comprises the steps of: at the device, generating first predicted sensor data based on the evaluation of the first run test. The method further comprises the steps of: at the device, evaluating a second run test of the second test matrix by processing the second sensor data using a second analysis model of the second aircraft system. The method further comprises the steps of: at the device, generating second predicted sensor data based on the evaluation of the second run test. The method further comprises the steps of: at the device, generating a first error measure by comparing a first subset of the sensor data with the first predicted sensor data. The method further comprises the steps of: at the device, generating a second error measure by comparing a second subset of the sensor data with the second predicted sensor data. The method further includes the step of determining a test coverage metric for a second test matrix based at least in part on the range of second sensor data. The method also includes the step of providing an output to a display device indicating a first error measure, a second error measure, and the test coverage metric.

[0006] In another specific implementation, an aircraft is provided, comprising an aircraft system, a memory, sensors, one or more processors, and a display device. The aircraft system includes a first aircraft system and a second aircraft system. The memory is configured to store multiple analysis models of the aircraft system. The multiple analysis models include a first analysis model of the first aircraft system and a second analysis model of the second aircraft system. The sensors are configured to generate sensor data during a first run test of a first test matrix of the first aircraft system. The sensor data includes first sensor data within a range specified by the first test matrix and second sensor data not specified by the first test matrix. The one or more processors are configured to evaluate the first run test by processing the first sensor data using the first analysis model. The one or more processors are further configured to generate first predictive sensor data based on the evaluation of the first run test. The one or more processors are further configured to evaluate a second run test of a second test matrix of the second aircraft system by processing the second sensor data using the second analysis model. The one or more processors are further configured to generate second predictive sensor data based on the evaluation of the second run test. The one or more processors are further configured to generate a first error measure by comparing a first subset of sensor data with first predictive sensor data. The one or more processors are also configured to generate a second error measure by comparing a second subset of sensor data with second predictive sensor data. The one or more processors are further configured to determine a test coverage metric for the second test matrix based at least in part on the range of the second sensor data. The display device is configured to display an output indicating the first error measure, the second error measure, and the test coverage metric.

[0007] In another particular implementation, a computer-readable storage device is provided storing instructions that, when executed by one or more processors, cause the one or more processors to obtain a first test matrix for a first aircraft system and a second test matrix for a second aircraft system. The instructions also cause the one or more processors, during a first run test of the first test matrix, to obtain sensor data including first sensor data within a range specified by the first test matrix and second sensor data not specified by the first test matrix. The instructions further cause the one or more processors to evaluate the first run test by processing the first sensor data using a first analysis model of the first aircraft system. The instructions further cause the one or more processors to generate first predictive sensor data based on the evaluation of the first run test. The instructions further cause the one or more processors to evaluate a second run test of the second test matrix by processing the second sensor data using a second analysis model of the second aircraft system. The instructions further cause the one or more processors to generate second predictive sensor data based on the evaluation of the second run test. The instructions further cause the one or more processors to generate a first error measure by comparing a first subset of the sensor data with the first predictive sensor data. The instruction further causes the one or more processors to generate a second error measure by comparing a second subset of sensor data with second predictive sensor data. The instruction further causes the one or more processors to determine a test coverage metric for the second test matrix based at least in part on the range of the second sensor data. The instruction further causes the one or more processors to provide an output to a display device indicating the first error measure, the second error measure, and the test coverage metric.

[0008] The features, functions, and advantages described herein can be implemented independently in various implementations or combined in other implementations. Further details can be found in the following description and figures. Attached Figure Description

[0009] Figure 1 This is a diagram illustrating a system configured to perform operational tests on an aircraft system.

[0010] Figure 2 yes Figure 1 A diagram showing an example of an analyzer.

[0011] Figure 3 This is a flowchart illustrating an example of an operational testing method for an aircraft system.

[0012] Figure 4 This is a flowchart illustrating the life cycle of an aircraft configured to perform operational tests of its flight system.

[0013] Figure 5 This is a block diagram illustrating an aircraft configured to perform operational tests of an aircraft system.

[0014] Figure 6 This is a block diagram of a computing environment according to the present disclosure, including aspects of a computing device configured to support computer-implemented methods and computer-executable program instructions (or code). Detailed Implementation

[0015] The aspects disclosed herein present operational testing of an aircraft system. For example, an aircraft includes one or more sensors that monitor the aircraft's condition. An analyzer can use a test matrix for the aircraft system. For example, a first test matrix for a first aircraft system indicates a set of operational tests for that first aircraft system. As an illustrative example, the first aircraft system includes the aircraft's fuel system, and a second operational test is used to determine the impact of altitude changes on fuel consumption. During the execution of the first operational test within the first test matrix, sensor data is collected from the sensors. For example, the sensor data includes first sensor data specified by the first operational test (e.g., detected altitude measurements). The sensor data also includes other sensor data not specified by the first test matrix. As an illustrative example, the sensor data includes detected engine temperature measurements.

[0016] The analyzer evaluates a first operational test by processing first sensor data (e.g., detected altitude) using a first analytical model of a first aircraft system (e.g., a fuel system). For example, the analyzer generates first predicted sensor data (e.g., predicted fuel consumption) by processing the first sensor data (e.g., detected altitude) using the first analytical model. The analyzer generates a first error measure by comparing a first subset of the sensor data (e.g., detected fuel consumption) with the first predicted sensor data (e.g., predicted fuel consumption).

[0017] Additionally, analytical models of other aircraft systems are used to process sensor data generated during the first operational test. For example, the analyzer evaluates a second operational test (e.g., mapping airspeed to engine temperature) using a second analytical model of a second aircraft system (e.g., an engine). To illustrate, the analyzer uses a second analytical model to process second sensor data (e.g., detected engine temperature) to generate second predictive sensor data (e.g., predicted engine temperature).

[0018] The analyzer generates a second error measure by comparing a second subset of sensor data (e.g., detected engine temperature) with second predicted sensor data (e.g., predicted engine temperature). Therefore, a portion of the second test matrix can be covered during the execution of a first run test of the first test matrix, without having to execute the second run test separately. In a specific example, the first run test can be scheduled after changes in the aircraft that are expected to affect the first aircraft system but not the second aircraft system. The analyzer can use the sensor data generated during the operation of the first aircraft system to evaluate the second run test and can detect the unintended effects of the changes in the aircraft on the second aircraft system.

[0019] The accompanying drawings and the following description illustrate specific exemplary embodiments. It should be appreciated that those skilled in the art will be able to design various arrangements that, while not expressly described or shown herein, embody the principles described herein and are included within the scope of the claims following this description. Furthermore, any examples described herein are intended to aid in understanding the principles of this disclosure and should be construed as not limiting. Consequently, this disclosure is not limited to the specific embodiments or examples described below, but is limited by the claims and their equivalents.

[0020] As used herein, various terms are used only for the purpose of describing a particular implementation and are not intended to be restrictive. For example, unless the context explicitly indicates otherwise, singular descriptions are also intended to include plural forms. Furthermore, some features described herein are singular in some implementations and plural in others. For example, Figure 1 It describes a configuration including one or more processors 128 ( Figure 1 The term "processor" in the context of System 100 indicates that in some implementations System 100 includes a single processor 128 while in other implementations System 100 includes multiple processors 128. For ease of reference, such features are generally referred to herein as "one or more" features, and are subsequently referred to in the singular unless an aspect relating to multiple features is being described.

[0021] The terms “comprise,” “comprises,” and “comprising” are used interchangeably with “include,” “includes,” or “including.” Additionally, the term “wherein” is used interchangeably with the term “where.” As used herein, “exemplary” indicates an example, implementation, and / or aspect, and should not be construed as limiting or indicating a preferred or preferred implementation. As used herein, ordinal terms used to modify elements (e.g., “first,” “second,” “third,” etc.) do not themselves indicate any priority or order of that element relative to another element, but rather distinguish that element from another element with the same name (but using ordinal terms). As used herein, the term “set” refers to a grouping of one or more elements, and the term “multiple” refers to multiple elements.

[0022] As used herein, unless the context otherwise indicates, “generate,” “calculate,” “use,” “select,” “access,” and “determine” are interchangeable. For example, “generate,” “calculate,” or “determine” a parameter (or signal) can refer to actively generating, calculating, or determining the parameter (or signal), or it can refer to using, selecting, or accessing the parameter (or signal) that has already been generated (e.g., generated by another component or device). As used herein, “coupled” can include “communicationally coupled,” “electrically coupled,” or “physically coupled,” and may also (or alternatively) include any combination of these. Two devices (or components) can be coupled directly or indirectly (e.g., communically coupled, electrically coupled, or physically coupled) via one or more other devices, components, wires, buses, networks (e.g., wired networks, wireless networks, or combinations thereof). As an illustrative, non-limiting example, two electrically coupled devices (or components) can be included in the same device or in different devices and can be connected via electronic devices, one or more connectors, or inductive coupling. In some implementations, such as in electrical communications, two devices (or components) connected communicatively can directly or indirectly (e.g., via one or more wires, buses, networks, etc.) send and receive electrical signals (digital or analog signals). As used herein, the term "direct connection" is used to describe two devices connected without intermediate components (e.g., communicatively, electrically, or physically).

[0023] Reference Figure 1A system configured to perform operational tests of an aircraft system is illustrated, and is generally designated as system 100. System 100 includes a device 102 that includes a memory 132 connected to one or more processors 128. In a particular aspect, device 102 includes one or more interfaces 134 configured to receive sensor data 151. In a particular aspect, interface 134 (e.g., a communication interface) includes at least one of a wired interface or a wireless interface. Memory 132 includes a computer-readable medium (e.g., a computer-readable storage device) storing instructions 116 executable by processor 128. Instructions 116 are executable to initiate, execute, or control operation to assist in performing operational tests of aircraft system 154.

[0024] In a particular example, memory 132 is configured to store an analysis model 111 of aircraft system 154. In a particular example, analysis model 111 includes at least one of the following: a physical model, an empirical model, a statistical model, a simulation model, or a combination thereof. In various examples, analysis model 111 includes: an engine model, a fuel system model, a nose wheel steering system model, a stabilizer system model, an air conditioning system model, a communication system model, a propulsion system model, an electrical system model, an environmental system model, a hydraulic system model, or a combination thereof. For illustration, analysis model 111 includes: a first analysis model 103 of a first aircraft system 156 (e.g., a fuel system), a second analysis model 113 of a second aircraft system 158 (e.g., an engine), and one or more additional analysis models of one or more additional aircraft systems, or a combination thereof. In a particular aspect, analysis model 111 is configured to predict aircraft system responses based on sensor data 151. For example, a first analytical model 103 is configured to predict the response of a first aircraft system 156 (e.g., a fuel system) as at least a portion of sensor data 151, a second analytical model 113 is configured to predict the response of a second aircraft system 158 (e.g., an engine) as at least a portion of sensor data 151, and one or more additional analytical models are configured to predict the responses of one or more additional aircraft systems as at least some of sensor data 151. Sensor data 151 is generated by one or more sensors 152 configured to monitor aircraft systems 154 of aircraft 104. In a particular aspect, sensor 152 includes one or more airborne sensors, one or more flight test sensors, or a combination thereof. In a particular aspect, one or more sensors 152 are employed in at least one of the following ways: coupled to aircraft 104, near aircraft, inside aircraft, outside aircraft, integrated into aircraft 104, or detached from aircraft.

[0025] In a particular aspect, memory 132 is configured to store a test matrix 131 for aircraft system 154. For example, test matrix 131 includes: a first test matrix 133 for a first aircraft system 156 (e.g., a fuel system), a second test matrix 173 for a second aircraft system 158 (e.g., an engine), one or more additional test matrices for one or more additional aircraft systems, or a combination thereof. First test matrix 133 describes one or more first operational tests 121. For example, first operational tests 121 include first operational tests 123. In a particular example, first operational tests 123 include testing the fuel consumption of the first aircraft system 156 (e.g., a fuel system) in response to changes in altitude. Second test matrix 173 describes one or more second operational tests 161. For example, second operational tests 161 include second operational tests 163. In a particular example, second operational tests 163 include testing the engine temperature of the second aircraft system 158 (e.g., an engine) in response to changes in airspeed. In some implementations, one or more additional test matrices describe one or more additional operational tests of one or more additional aircraft systems.

[0026] Processor 128 includes analyzer 130, which can be implemented at least in part by processor 128 executing instructions 116. Processor 128 can be implemented as a single processor or as multiple processors, such as in a multi-core configuration, a multi-processor configuration, a distributed computing configuration, a cloud computing configuration, or any combination thereof. In some implementations, one or more portions of analyzer 130 are implemented by processor 128 using dedicated hardware, firmware, or a combination of both.

[0027] Analyzer 130 includes an evaluator 135 connected to output generator 138 via error analyzer 136. Output generator 138 is connected to display device 120. Although device 102, aircraft 104, and display device 120 are illustrated as separate entities, in some implementations, two or more components of device 102, aircraft 104, or display device 120 are combined. For example, in a particular implementation, one or more components of display device 120, device 102, or a combination thereof are integrated into or on aircraft 104. Evaluator 135 is configured to evaluate a run test by processing sensor data using an analytical model. For example, evaluator 135 generates predicted sensor data based on the evaluation. Error analyzer 136 is configured to generate an error metric based on a comparison of predicted sensor data and a subset of sensor data. Output generator 138 is configured to generate an aircraft test output based on the error metric.

[0028] During operation, analyzer 130 obtains test matrix 131. For example, analyzer 130 obtains test matrix 131 from a test server during the pre-flight sequence. Aircraft 104 performs a first operational test 123. In a particular implementation, analyzer 130 sends a command to aircraft 104 to initiate the first operational test 123 (e.g., flying at various altitudes during a specific time period or during flight). In an alternative implementation, the pilot initiates the first operational test 123. In a particular aspect, the first operational test 123 includes at least one of ground tests, taxiing tests, or flight tests. These tests are to be performed while aircraft 104 is on the ground (e.g., parked), while aircraft 104 is taxiing, or during flight, respectively.

[0029] Sensor 152 generates sensor data 151 during the first operational test 123. For example, sensor 152 includes a first sensor, a second sensor, a third sensor, and a fourth sensor, which respectively monitor changes in altitude, changes in fuel consumption, changes in airspeed, and changes in engine temperature. For illustration, sensor data 151 includes sensor data 175 (e.g., altitude measurement result), sensor data 147 (e.g., fuel consumption measurement result), sensor data 153 (e.g., airspeed measurement result), sensor data 157 (e.g., engine temperature measurement result), or combinations thereof. Although sensor data 175, sensor data 147, sensor data 153, and sensor data 157... Figure 1 The data is exemplified as a different subset of sensor data 151, but in some examples, one or more of sensor data 175, sensor data 147, sensor data 153, or sensor data 157 may overlap.

[0030] Analyzer 130 obtains sensor data 151 from sensor 152. Sensor data 151 includes sensor data 175 (e.g., altitude measurement results) that are within a first range 145 (e.g., between a first altitude and a second altitude) specified by the first test matrix 133. Sensor data 151 also includes sensor data 153 (e.g., airspeed measurement results) that are not specified by the first test matrix 133.

[0031] Analyzer 130 evaluates one or more run tests by processing sensor data 151 based on analysis model 111. For example, evaluator 135 evaluates the first run test 123 in response to determining that sensor data 151 was generated during the first run test 123 by processing sensor data 175 (e.g., altitude measurement results) using the first analysis model 103, as further referenced. Figure 2As described. For illustration, evaluator 135 generates first predicted sensor data 141 based on the evaluation of the first operational test 123. In a particular example, in response to determining that the first operational test 123 tested the responsiveness of the first aircraft system 156 (e.g., fuel system) to changes in altitude, evaluator 135 provides sensor data 175 (e.g., altitude measurement results) as input to a first analysis model 103, and the first analysis model 103 determines the predicted response of the first aircraft system 156 to the sensor data 175. For illustration, the predicted response indicates predicted sensor data from various sensors. In a particular aspect, in response to determining that the first operational test 123 tested a change in fuel consumption of the first aircraft system 156 (e.g., fuel system), evaluator 135 extracts the first predicted sensor data 141 (e.g., predicted fuel consumption) from the predicted response.

[0032] Error analyzer 136 generates a first error measure 143 based on a comparison of first predictive sensor data 141 (e.g., predicted fuel consumption) and sensor data 147 (e.g., detected fuel consumption), as further referenced. Figure 2 As described. Output generator 138 determines a first test coverage metric 139 based at least in part on a first range 145 of sensor data 175 (e.g., detected height), as further referenced. Figure 2 As described. For example, the first test coverage metric 139 indicates how much of the first test matrix 133 has been covered. In a particular aspect, the output generator 138 determines the first test convergence metric 177 based on the first error metric 143, as further referenced. Figure 2 As described. The first test convergence metric 177 indicates whether the results of the first run test 121 evaluating the first test matrix 133 have converged. The output generator 138 generates a first aircraft test output 149, which indicates the first error metric 143, the first test coverage metric 139, the first test convergence metric 177, or a combination thereof.

[0033] In a particular aspect, output generator 138, in response to determining that a first error measure 143 does not meet an error criterion, generates a first aircraft test output 149 indicating that an error was detected in the first analysis model 103, the first aircraft system 156, or both. In a particular aspect, in response to determining that a first test coverage measure 139 meets a coverage criterion (e.g., at least 90% coverage), a first test convergence measure 177 meets a convergence criterion (e.g., standard deviation of convergence within a threshold range with 95% confidence), or both, output generator 138 generates the first aircraft test output 149 indicating test completion of the first analysis model 103, designates the first analysis model 103 as meeting the test completion criterion, or both.

[0034] Analyzer 130 uses sensor data 151 generated during the first run test 123 to evaluate one or more additional run tests. For example, in response to determining that the second analysis model 113 was not specified to meet the test completion criteria, evaluator 135 evaluates the second run test 163 by processing the sensor data 153 (e.g., airspeed measurement results) generated during the first run test 123 using the second analysis model 113, as further referenced. Figure 2 As described. For illustration, evaluator 135 generates second predicted sensor data 181 based on the evaluation of the second operational test 163. In a particular example, in response to determining that the second operational test 163 tested the responsiveness of the second aircraft system 158 (e.g., engine) to changes in airspeed, evaluator 135 provides sensor data 153 (e.g., airspeed measurement results) as input to a second analysis model 113, and the second analysis model 113 determines the predicted response of the second aircraft system 158 to the sensor data 153. For illustration, the predicted response indicates predicted sensor data from various sensors. In a particular aspect, in response to determining that the second operational test 163 tested a change in engine temperature of the second aircraft system 158 (e.g., engine), evaluator 135 extracts the second predicted sensor data 181 (e.g., the predicted engine temperature) from the predicted response.

[0035] Error analyzer 136 generates a second error measure 183 based on a comparison of second predicted sensor data 181 (e.g., predicted engine temperature) and sensor data 157 (e.g., detected engine temperature), as further referenced. Figure 2 As described, the output generator 138 determines the second test coverage metric 197 based at least in part on a second range 195 of sensor data 153 (e.g., detected engine temperature), as further referenced. Figure 2As described. For example, the second test coverage metric 197 indicates how much the second test matrix 173 has been covered. In a particular aspect, the output generator 138 determines the second test convergence metric 179 based on the second error metric 183, as further referenced. Figure 2 As described. The second test convergence metric 179 indicates whether the results of the second run test 161 evaluating the second test matrix 173 have converged. The output generator 138 generates a second aircraft test output 199, which indicates a second error metric 183, a second test coverage metric 197, a second test convergence metric 179, or a combination thereof.

[0036] In a specific aspect, output generator 138, in response to determining that the second error metric 183 does not meet the error criterion, generates a second aircraft test output 199 indicating that an error was detected in the second analysis model 113, the second aircraft system 158, or both. In a specific aspect, output generator 138, in response to determining that the second test coverage metric 197 meets the coverage criterion (e.g., at least 90% coverage), the second test convergence metric 179 meets the convergence criterion (e.g., standard deviation of convergence within a threshold range with 95% confidence), or both, generates a second aircraft test output 199 indicating the test completion of the second analysis model 113, designates the second analysis model 113 as meeting the test completion criterion, or both. Similarly, analyzer 130 can use sensor data 151 generated during the first run test 123 to evaluate one or more additional run tests. In a specific implementation, fewer run tests are evaluated over time because more analysis models are designated as meeting the test completion criterion.

[0037] In some implementations, analyzer 130, in response to determining that an aircraft system has been updated, resets the coverage and convergence metrics for the analysis model of one or more aircraft systems 154. For example, in response to determining that a second aircraft system 158 has been updated for the first time, analyzer 130 resets the second test coverage metric 197 to a default value (e.g., indicating no coverage), resets the second test convergence metric 179 to a default value (e.g., indicating no convergence), or both. In a particular implementation, analyzer 130, in response to determining that a second aircraft system 158 has been updated for the first time, resets the first test coverage metric 139 to a default value (e.g., indicating no coverage), resets the first test convergence metric 177 to a default value (e.g., indicating no convergence), or both. Therefore, the running tests of the analysis model can be restarted after the update.

[0038] Output generator 138 provides a first aircraft test output 149, a second aircraft test output 199, or both to display device 120. In a specific example, user 118 may submit a first analysis model 103 or a second analysis model 113 that has met the test completion criteria for certification.

[0039] Therefore, system 100 enables the automatic evaluation of one or more running tests (e.g., a second running test 163) based on sensor data 151 generated during the execution of another running test (e.g., a first running test 123). For example, sensor data 151 generated during the first running test 123 can be automatically used to evaluate multiple running tests that have not yet met the test completion criteria. In a particular example, the first running test 123 is performed after a change in aircraft 104 that did not have an expected impact on the second aircraft system 158. Automatic evaluation of the second running test 163 can help identify any unintended impacts on the second aircraft system 158. For example, output generator 138 can generate a second aircraft test output 199 in response to determining that a second error measure 183 does not meet the error criteria, indicating errors in the second analysis model 113, the second aircraft system 158, or both.

[0040] Although the evaluator 135, error analyzer 136, and output generator 138 are described as separate components, in other implementations, the described functions of two or more of the evaluator 135, error analyzer 136, and output generator 138 may be performed by a single component. In some implementations, one or more of each of the evaluator 135, error analyzer 136, output generator 138, analyzer 130, processor 128, device 102, aircraft 104, or display device 120 may be represented in hardware, such as via an application-specific integrated circuit (ASIC) or field-programmable gate array (FPGA), or the operations described with reference to these components may be performed by a processor executing computer-readable instructions.

[0041] Reference Figure 2 An example of analyzer 130 is shown. In the first example, running test 223, analyzing model 213, sensor data 253, predicting sensor data 241, sensor data 257, error metric 243, range 245, test matrix 225, test coverage metric 247, test convergence metric 249, and aircraft test output 251 correspond to respectively Figure 1The first run test 123, first analysis model 103, sensor data 175, first predicted sensor data 141, sensor data 147, first error measure 143, first range 145, first test matrix 133, first test coverage measure 139, first test convergence measure 177, and first aircraft test output 149 are respectively. In the second example, run test 223, analysis model 213, sensor data 253, predicted sensor data 241, sensor data 257, error measure 243, range 245, test matrix 225, test coverage measure 247, test convergence measure 249, and aircraft test output 251 correspond to respectively Figure 1 The second operational test 163, the second analysis model 113, sensor data 153, the second predictive sensor data 181, sensor data 157, the second error measure 183, the second range 195, the second test matrix 173, the second test coverage measure 197, the second test convergence measure 179, and the second aircraft test output 199.

[0042] During operation, evaluator 135 evaluates the operational test 223 of test matrix 225 by processing sensor data 253 using analysis model 213. Based on the evaluation, evaluator 135 generates predictive sensor data 241. For example, in response to changes in the operational test 223 of a specific aircraft system 154, evaluator 135 processes the first type of sensor data 253 using the analysis model 213 for that specific aircraft system to generate the second type of predictive sensor data 241.

[0043] Error analyzer 136 generates error measure 243 based on a comparison of sensor data 257 (e.g., detected second-type sensor data) with predicted sensor data 241. In a particular example, error measure 243 indicates the difference, ratio, or both between sensor data 257 and predicted sensor data 241.

[0044] Output generator 138 determines a range 245 (e.g., minimum and maximum values ​​indicated by the sensor data) of sensor data 253 generated by the first sensor in sensor 152. Output generator 138 determines a test coverage metric 247 of test matrix 225 based on range 245. For example, test matrix 225 indicates a first data range 261 of the first sensor indicated by test matrix 225 (e.g., first altitude to second altitude or first airspeed to second airspeed). For illustration, test matrix 225 indicates that the response of a particular aircraft system should be tested against a first data range of the first sensor (e.g., first range 145). Output generator 138 updates test coverage metric 247 to indicate that the data range 245 of the first sensor has been tested.

[0045] In a particular aspect, analyzer 130 receives a set of sensor data during a data collection period. In a particular example, the data collection period is initiated in response to a recent update of a particular aircraft system, a recent update of any of the aircraft systems 154, a recent update of another particular aircraft system 154, or a combination thereof. In a particular aspect, the data collection period is initiated in response to receiving user input.

[0046] In a specific aspect, analyzer 130 updates the test coverage metric 247 during the data collection period based on the sensor data set. For example, analyzer 130 determines the range 245 of sensor data 253 generated by the first sensor during a first time period of the data collection period, and determines a second range 263 of sensor data generated by the first sensor during a second time period of the data collection period. In a specific aspect, the first time period is the execution period of run test 223, and the second time period is the execution period of another run test of test matrix 225. Analyzer 130 updates the test coverage metric 247 to indicate that range 245 and second range 263 have been tested from the first range 261 indicated by test matrix 225. In a specific aspect, output generator 138 updates the test coverage metric 247 to indicate the proportion (e.g., percentage) of the first range 261 that has been tested during the data collection period.

[0047] In a specific aspect, output generator 138 determines a test convergence metric 249 based on the set of sensor data received during the data collection period. For example, output generator 138 determines test convergence metric 249 to indicate whether an error measure corresponding to the set of sensor data is converging. For illustration, error analyzer 136 determines an error measure 243 corresponding to sensor data 253 generated by the first sensor during a first time period, and determines a second error measure corresponding to sensor data generated by the first sensor during a second time period.

[0048] Output generator 138 processes error measure 243, a second error measure, one or more additional error measures corresponding to one or more additional sensor data sets, or combinations thereof, using statistical methods to determine test convergence metric 249. For example, output generator 138 updates test convergence metric 249 to indicate the standard deviation calculated based on error measure 243, the second error measure, additional error measures, or combinations thereof. In a particular aspect, output generator 138 updates test convergence metric 249 to indicate the confidence interval associated with the standard deviation.

[0049] Output generator 138 generates aircraft test output 251 based on error metric 243, test coverage metric 247, test convergence metric 249, or a combination thereof. In a particular aspect, output generator 138 generates aircraft test output 251 in response to determining that error metric 243 does not meet (e.g., is greater than or equal to) an error criterion (e.g., an error threshold) to indicate that an error was detected in analysis model 213, a specific aircraft system, or both.

[0050] In a specific aspect, output generator 138, in response to determining that test coverage metric 247 meets coverage criteria, test convergence metric 249 meets convergence metric, or both, generates aircraft test output 251 indicating that the test of analysis model 213 is complete.

[0051] In a specific example, analyzer 130 evaluates the run test 223 of test matrix 225 based on sensor data 253 generated during the execution of run test 223 or during the execution of another run test of test matrix 225. In an alternative example, analyzer 130 evaluates the run test 223 of test matrix 225 without executing the run test of test matrix 225. In this example, analyzer 130 evaluates run test 223 based on sensor data 253 generated during the execution of another run test of another test matrix. Therefore, analyzer 130 enables coverage of at least a portion of test matrix 225 based on sensor data generated during the execution of run tests not included in test matrix 225.

[0052] Reference Figure 3 This illustrates an example of an operational testing method for an aircraft system, and is generally designated as Method 300. In certain aspects, one or more operations of Method 300 are performed by... Figure 1 The evaluator 135, error analyzer 136, output generator 138, analyzer 130, processor 128, device 102, aircraft 104, system 100, or a combination thereof, are used to perform this function.

[0053] Method 300 includes the following steps: at 302, obtaining a first test matrix for a first aircraft system and a second test matrix for a second aircraft system. For example, Figure 1 The analyzer 130 obtains the first test matrix 133 of the first aircraft system 156 and the second test matrix 173 of the second aircraft system 158.

[0054] The method 300 further includes the step of: at 304, during a first run test of the first test matrix, acquiring sensor data including first sensor data within a range specified by the first test matrix and second sensor data not specified by the first test matrix. For example, Figure 1 During the first run test 123 of the first test matrix 133, the evaluator 135 acquires sensor data 151, which includes sensor data 175 (e.g., detected altitude) within a first range 145 (e.g., within a first range 261) specified by the first test matrix 133, and also includes sensor data 153 (e.g., detected airspeed) not specified by the first test matrix 133, as shown in reference. Figures 1 to 2 As described.

[0055] Method 300 further includes the step of: at 306, evaluating the first operational test by processing the first sensor data using a first analytical model of the first aircraft system. For example, Figure 1 The evaluator 135 evaluates the first operational test 123 by processing sensor data 175 using the first analysis model 103 of the first aircraft system 156, as shown in reference. Figures 1 to 2 As described.

[0056] Method 300 further includes the step of: at 308, generating first predictive sensor data based on the evaluation of the first run test. For example, Figure 1 The evaluator 135 generates first predictive sensor data 141 based on the evaluation of the first run test 123, as shown in reference. Figures 1 to 2 As described.

[0057] Method 300 further includes the step of: at 310, evaluating a second operational test of the second test matrix by processing the second sensor data using a second analytical model of the second aircraft system. For example, Figure 1 The evaluator 135 evaluates the second run test 163 of the second test matrix 173 by processing sensor data 153 using the second analysis model 113 of the second aircraft system 158, as shown in reference. Figures 1 to 2 As described.

[0058] Method 300 further includes the step of: at 312, generating second predictive sensor data based on the evaluation of the second run test. For example, Figure 1 The evaluator 135 generates second predictive sensor data 181 based on the evaluation of the second run test 163, as shown in reference. Figures 1 to 2 As described.

[0059] Method 300 further includes the step of: at 314, generating a first error measure by comparing a first subset of sensor data with first predictive sensor data. For example, Figure 1 The error analyzer 136 generates a first error measure 143 by comparing sensor data 147 with first predicted sensor data 141, as shown in reference. Figures 1 to 2 As described.

[0060] Method 300 further includes the step of: at 316, generating a second error measure by comparing a second subset of the sensor data with second predicted sensor data. For example, Figure 1 The evaluator 135 generates a second error measure 183 by comparing sensor data 157 with second predicted sensor data 181, as shown in reference. Figures 1 to 2 As described.

[0061] Method 300 further includes the step of: at 318, determining a test coverage metric for the second test matrix based at least in part on the range of the second sensor data. For example, Figure 1 The output generator 138 determines the second test coverage metric 197 of the second test matrix 173 based at least in part on the second range 195 of the sensor data 153, as referenced. Figures 1 to 2 As described.

[0062] Method 300 further includes the step of: at 320, providing an output to a display device, the output indicating a first error measure, a second error measure, and a test coverage measure. For example, Figure 1 The output generator 138 provides the first aircraft test output 149, the second aircraft test output 199, or both to the display device 120. The first aircraft test output 149, the second aircraft test output 199, or both, as shown in reference... Figures 1 to 2 As described.

[0063] Therefore, method 300 enables the automatic evaluation of the second run test 163 based on sensor data 151 generated during the execution of the first run test 123. In a specific example, the sensor data 151 generated during the first run test 123 can be automatically used to evaluate multiple run tests with multiple matrices using an analytical model of multiple aircraft systems.

[0064] Reference Figure 4 An exemplary flowchart of the life cycle of an aircraft, including an analyzer 130, is shown and designated as method 400. During pre-production, this exemplary method 400 includes the following steps, at 402, the aircraft (such as reference...) Figure 5 The description of the aircraft 500 includes its specifications and design. During the specification and design of the aircraft, method 400 may include the following steps: specification and design of analyzer 130. At 404, method 400 includes the following step: material procurement, which may include procuring materials for analyzer 130.

[0065] During production, method 400 includes the following steps: at 406, component and sub-component manufacturing, and at 408, system integration of the aircraft. For example, method 400 may include the steps of: component and sub-component manufacturing of analyzer 130 and system integration of analyzer 130. At 410, method 400 includes the steps of: aircraft certification and delivery, and at 412, putting the aircraft into use. In a particular aspect, certification and delivery may include operational testing of the aircraft. Certification and delivery may include certification of analyzer 130 to put analyzer 130 into use. When used by a customer, routine maintenance and servicing may be scheduled for the aircraft (this may also include modification, reconfiguration, refurbishment, operational testing, etc.). At 414, method 400 includes the step of: performing maintenance and servicing on the aircraft, which may include performing maintenance and servicing on analyzer 130. In a particular aspect, maintenance and servicing of the aircraft may include operational testing.

[0066] Each process in method 400 can be performed or implemented by a system integrator, a third party, and / or an operator (e.g., a customer). For the purposes of this description, a system integrator can include, but is not limited to, any number of aircraft manufacturers and main system subcontractors; a third party can include, without limitation, any number of vendors, subcontractors, and suppliers; and an operator can be an airline, leasing company, military entity, service organization, etc.

[0067] Various aspects of this disclosure can be described within the context of examples of vehicles. Specific examples of vehicles are as follows: Figure 5 The aircraft shown is 500.

[0068] exist Figure 5 In the example, aircraft 500 includes a frame 518 with aircraft system 154, sensor 152, and interior 522. Examples of aircraft system 154 may include one or more of the following: propulsion system 524, electrical system 526, environmental system 528, hydraulic system 530, and analyzer 130. Any number of other systems may be included. In a particular aspect, aircraft 500 corresponds to... Figure 1 The aircraft 104.

[0069] Figure 6 This is a block diagram of a computing environment 600 comprising a computing device 610 configured to support computer-implemented methods and computer-executable program instructions (or code) according to this disclosure. For example, the computing device 610 or a portion thereof is configured to initiate, execute, or control references. Figures 1 to 6 Instructions describing one or more operations.

[0070] The computing device 610 includes one or more processors 620. In a particular aspect, the processor 620 corresponds to Figure 1 The processor 128. The processor 620 is configured to communicate with system memory 630, one or more storage devices 640, one or more input / output interfaces 650, one or more communication interfaces 660, or any combination thereof. System memory 630 includes volatile memory devices (e.g., random access memory (RAM) devices), non-volatile memory devices (e.g., read-only memory (ROM) devices, programmable read-only memory, and flash memory), or both. System memory 630 stores operating system 632, which may include a basic input / output system for booting computing device 610 and a complete operating system enabling computing device 610 to interact with users, other programs, and other devices. System memory 630 stores system (program) data 636, such as... Figure 1 The analysis model 111, test matrix 131, sensor data 151, first predicted sensor data 141, second predicted sensor data 181, first error measure 143, second error measure 183, first range 145, second range 195, first test coverage measure 139, second test coverage measure 197, first test convergence measure 177, second test convergence measure 179, first aircraft test output 149, second aircraft test output 199, and display device 120 are included. Figure 2 The operational test 223, sensor data 253, analysis model 213, predicted sensor data 241, sensor data 257, error measure 243, test matrix 225, range 245, test coverage measure 247, test convergence measure 249, aircraft test output 251, first range 261, second range 263, or a combination thereof.

[0071] System memory 630 includes one or more applications 634 (e.g., instruction sets) executable by processor 620. As an example, the one or more applications 634 include those executable by processor 620 to initiate, control, or execute references. Figures 1 to 6Instructions describing one or more operations. For example, the one or more applications 634 include instructions executable by processor 620 to initiate, control, or perform one or more operations described by reference analyzer 130.

[0072] In a particular implementation, system memory 630 includes a non-transitory computer-readable medium (e.g., a computer-readable storage device) storing instructions that, when executed by processor 620, cause processor 620 to initiate, execute, or control operations for performing or analyzing operational tests of an aircraft system. In a particular aspect, the instructions cause processor 620 to obtain a first test matrix for a first aircraft system and a second test matrix for a second aircraft system. The instructions also cause processor 620, during a first operational test of the first test matrix, to obtain sensor data including first sensor data within a range specified by the first test matrix and second sensor data not specified by the first test matrix. The instructions further cause processor 620 to evaluate the first operational test by processing the first sensor data using a first analysis model of the first aircraft system. The instructions further cause processor 620 to generate first predictive sensor data based on the evaluation of the first operational test. The instructions further cause processor 620 to evaluate a second operational test of the second test matrix by processing the second sensor data using a second analysis model of the second aircraft system. The instructions further cause processor 620 to generate second predictive sensor data based on the evaluation of the second operational test. The instruction further causes processor 620 to generate a first error measure by comparing a first subset of sensor data with first predictive sensor data. The instruction further causes processor 620 to generate a second error measure by comparing a second subset of sensor data with second predictive sensor data. The instruction further causes processor 620 to determine a test coverage metric for a second test matrix based at least in part on the range of the second sensor data. The instruction further causes processor 620 to provide output to a display device indicating the first error measure, the second error measure, and the test coverage metric.

[0073] In a particular aspect, each of the first and second operational tests includes at least one of ground testing, taxiing testing, and flight testing. In a particular aspect, the instruction causes processor 620 to determine that a second test matrix indicates a first data range from the first sensor. The instruction also causes processor 620 to determine a test coverage metric, at least in part, based on a third data range that is included within the first data range but not within the second data range, in response to determining that a second data range from the first sensor has been processed using a second analysis model during the data collection period. The range of the second sensor data includes the second data range from the first sensor.

[0074] In a particular aspect, the instruction causes processor 620 to acquire a set of sensor data during a data collection period. The set of sensor data includes the sensor data. The instruction also causes processor 620 to evaluate a plurality of second run tests of a second test matrix based on a second analysis model and the set of sensor data. The plurality of second run tests includes the second run test. The instruction further causes processor 620 to determine a plurality of second error measures based on the evaluation of the plurality of second run tests. The plurality of second error measures includes the second error measure. The instruction further causes processor 620 to determine a test convergence metric of the second test matrix based on the plurality of second error measures. In a particular aspect, the instruction causes processor 620 to generate an output indicating that the test of the second analysis model is complete in response to determining that a second error measure does not meet an error metric. In a particular aspect, the instruction causes processor 620 to generate an output indicating that an error was detected in the second analysis model, the second aircraft system, or both in response to determining that a second error measure does not meet an error metric.

[0075] The one or more storage devices 640 include non-volatile storage devices, such as disks, optical disks, or flash memory devices. In a particular example, storage device 640 includes both removable and non-removable storage devices. Storage device 640 is configured to store an operating system, an image of the operating system, applications (e.g., one or more applications of application 634), and program data (e.g., program data 636). In a particular aspect, system memory 630, storage device 640, or both include tangible computer-readable media. In a particular aspect, one or more storage devices of storage device 640 are external to computing device 610. In a particular aspect, system memory 630 includes... Figure 1 The memory 132.

[0076] The one or more input / output interfaces 650 enable the computing device 610 to communicate with one or more input / output devices 670 to facilitate user interaction. For example, the one or more input / output interfaces 650 may include a display interface, an input interface, or both. For example, the input / output interface 650 is adapted to receive input from a user, input from another computing device, or a combination thereof. In some implementations, the input / output interface 650 conforms to one or more standard interface protocols, including serial interfaces (e.g., Universal Serial Bus (USB) interfaces or Institute of Electrical and Electronics Engineers (IEEE) interface standards), parallel interfaces, display adapters, audio adapters, or custom interfaces (“IEEE” is a registered trademark of The Institute of Electrical and Electronics Engineers, Inc. of Piscataway, New Jersey). In some implementations, the input / output device 670 includes one or more user interface devices and displays, including a combination of buttons, keyboards, pointing devices, displays, speakers, microphones, touchscreens, and other devices. In a particular aspect, the input / output device 670 includes... Figure 1 The display device 120.

[0077] The processor 620 is configured to communicate with the device or controller 680 via one or more communication interfaces 660. For example, the one or more communication interfaces 660 may include a network interface, such as... Figure 1 Interface 134. Device or controller 680 may, for example, include... Figure 1 The aircraft 104, one or more other devices, or any combination thereof.

[0078] In some implementations, a non-transitory computer-readable medium is provided that stores instructions, when executed by one or more processors, cause the processors to initiate, execute, or control operations for performing some or all of the functions described above. For example, the instructions may be executable to implement... Figures 1 to 6 One or more operations or methods. In some implementations, Figures 1 to 6 One or more of the operations or methods may be implemented by one or more processors executing instructions (e.g., one or more central processing units (CPUs), one or more graphics processing units (GPUs), one or more digital signal processors (DSPs)), by dedicated hardware circuitry, or any combination of both.

[0079] This disclosure includes examples pursuant to the following terms:

[0080] Clause 1. A testing method, said method comprising the following steps:

[0081] At the device, the first test matrix of the first aircraft system and the second test matrix of the second aircraft system are obtained;

[0082] During the first run test of the first test matrix, sensor data is obtained, the sensor data including first sensor data within the range specified by the first test matrix and second sensor data not specified by the first test matrix;

[0083] At the device, the first operational test is evaluated by processing the first sensor data using a first analysis model of the first aircraft system;

[0084] At the device, first predictive sensor data is generated based on the evaluation of the first operational test;

[0085] At the device, a second operational test of the second test matrix is ​​evaluated by processing the second sensor data using a second analysis model of the second aircraft system;

[0086] At the device, second predictive sensor data is generated based on the evaluation of the second operational test;

[0087] At the device, a first error measure is generated by comparing a first subset of the sensor data with the first predicted sensor data;

[0088] At the device, a second error measure is generated by comparing a second subset of the sensor data with the second predicted sensor data;

[0089] The test coverage metric of the second test matrix is ​​determined at least in part based on the range of the second sensor data; and

[0090] An output (320) is provided to a display device, the output indicating the first error measure, the second error measure, and the test coverage measure.

[0091] Clause 2. The method according to Clause 1, wherein the first operational test includes at least one of ground testing, taxiing testing and flight testing.

[0092] Clause 3. The method described in accordance with Clause 1 or 2, wherein the first analytical model includes a physical model, an empirical model, a statistical model, a simulation model, or a combination thereof.

[0093] Clause 4. The method described according to Clause 1, 2 or 3 further comprises the following steps:

[0094] The second test matrix is ​​determined to indicate the first data range of the first sensor; and

[0095] In response to determining that the second data range from the first sensor has been processed using the second analysis model during the data collection period, the test coverage metric is determined at least in part based on a third data range, wherein the third data range is included in the first data range but not included in the second data range, and wherein the range of the second sensor data includes the second data range from the first sensor.

[0096] Clause 5. The method according to any one of Clauses 1 to 4, the method further comprising the step of: determining a test convergence metric of the second test matrix based at least in part on the second error metric, wherein the output indicates the test convergence metric.

[0097] Clause 6. The method described pursuant to any of Clauses 1 to 5, the method further comprising the following steps:

[0098] A set of sensor data is received during the data collection period, wherein the set of sensor data includes the sensor data;

[0099] The second test matrix is ​​evaluated based on the second analysis model and the sensor data set, wherein the multiple second run tests include the second run test;

[0100] Multiple second error measures are determined based on the evaluation of the multiple second operational tests, wherein the multiple second error measures include the second error measures; and

[0101] The test convergence metric of the second test matrix is ​​determined based on the plurality of second error measures.

[0102] Clause 7. The method according to Clause 6, wherein the test convergence metric is based at least in part on the standard deviation of the plurality of second error measures.

[0103] Clause 8. The method according to any one of Clauses 1 to 7, the method further comprising the step of: generating the output in response to determining that the test coverage metric satisfies the coverage criterion and the test convergence metric of the second test matrix satisfies the convergence criterion, to indicate that the test of the second analysis model is complete.

[0104] Clause 9. The method according to any one of Clauses 1 to 8, the method further comprising the step of: generating the output in response to determining that the second error measure does not meet the error criterion, to indicate that an error has been detected in the second analysis model, the second aircraft system, or both.

[0105] Clause 10. An aircraft comprising:

[0106] The aircraft system (154) includes a first aircraft system and a second aircraft system;

[0107] A memory (132, 630) configured to store multiple analysis models (111) of the aircraft system (154), the multiple analysis models (111) including a first analysis model of the first aircraft system and a second analysis model of the second aircraft system;

[0108] A sensor, configured to generate sensor data during a first operational test of a first test matrix of the first aircraft system, wherein the sensor data includes first sensor data within a range specified by the first test matrix and second sensor data not specified by the first test matrix.

[0109] One or more processors, said one or more processors being configured to:

[0110] The first operational test is evaluated by processing the first sensor data using the first analysis model.

[0111] First predictive sensor data is generated based on the evaluation of the first run test;

[0112] The second operational test of the second test matrix of the second aircraft system is evaluated by processing the second sensor data using the second analysis model.

[0113] The second predictive sensor data is generated based on the evaluation of the second run test;

[0114] A first error measure is generated by comparing a first subset of the sensor data with the first predictive sensor data.

[0115] A second error measure is generated by comparing a second subset of the sensor data with the second predicted sensor data; and

[0116] The test coverage metric of the second test matrix is ​​determined at least in part based on the range of the second sensor data; and

[0117] A display device is configured to display an output indicating the first error measure, the second error measure, and the test coverage measure.

[0118] Clause 11. The aircraft as described in Clause 10, wherein the sensor includes an airborne sensor, a flight test sensor, or a combination thereof.

[0119] Clause 12. The aircraft as described in Clause 10 or 11, wherein the second analytical model comprises a physical model, an empirical model, a statistical model, a simulation model, or a combination thereof.

[0120] Clause 13. The aircraft according to Clause 10, 11 or 12, wherein the one or more processors are further configured to: generate the output in response to determining that the test coverage metric satisfies the coverage criterion and that the test convergence metric of the second test matrix satisfies the convergence criterion, to indicate that the test of the second analysis model is complete.

[0121] Clause 14. The aircraft according to any one of Clauses 10 to 13, wherein the one or more processors are further configured to: generate the output in response to determining that the second error measure does not meet the error criterion, to indicate that an error has been detected in the second analysis model, the second aircraft system, or both.

[0122] Clause 15. A computer-readable storage device for storing instructions, said instructions, when executed by one or more processors, causing said one or more processors to:

[0123] Obtain the first test matrix of the first aircraft system and the second test matrix of the second aircraft system;

[0124] During the first run test of the first test matrix, sensor data is obtained, the sensor data including first sensor data within the range specified by the first test matrix and second sensor data not specified by the first test matrix;

[0125] The first operational test is evaluated by processing the first sensor data using a first analysis model of the first aircraft system.

[0126] First predictive sensor data is generated based on the evaluation of the first run test;

[0127] The second operational test of the second test matrix is ​​evaluated by processing the second sensor data using the second analysis model of the second aircraft system.

[0128] The second predictive sensor data is generated based on the evaluation of the second run test;

[0129] A first error measure is generated by comparing a first subset of the sensor data with the first predictive sensor data.

[0130] A second error measure is generated by comparing a second subset of the sensor data with the second predicted sensor data.

[0131] The test coverage metric of the second test matrix is ​​determined at least in part based on the range of the second sensor data; and

[0132] An output is provided to a display device, the output indicating the first error measure, the second error measure, and the test coverage measure.

[0133] Clause 16. The computer-readable storage device according to Clause 15, wherein each of the first operational test and the second operational test includes at least one of a ground test, a taxiing test, and a flight test.

[0134] Clause 17. A computer-readable storage device according to Clause 15 or 16, wherein the instructions further cause the one or more processors to:

[0135] The second test matrix is ​​determined to indicate the first data range (261) of the first sensor; and

[0136] In response to determining that the second data range from the first sensor has been processed using the second analysis model during the data collection period, the test coverage metric is determined at least in part based on a third data range (263), wherein the third data range is included within the first data range (261) but not within the second data range, wherein the range of the second sensor data includes the second data range from the first sensor.

[0137] Clause 18. A computer-readable storage device according to Clause 15, 16, or 17, wherein the instructions further cause the one or more processors to:

[0138] A set of sensor data is obtained during the data collection period, wherein the set of sensor data includes the sensor data;

[0139] The second test matrix is ​​evaluated based on the second analysis model and the sensor data set, wherein the multiple second run tests include the second run test;

[0140] Multiple second error measures are determined based on the evaluation of the multiple second operational tests, wherein the multiple second error measures include the second error measures; and

[0141] The test convergence metric of the second test matrix is ​​determined based on the plurality of second error measures.

[0142] Clause 19. A computer-readable storage device according to any one of Clauses 15 to 18, wherein the instructions further cause the one or more processors to generate the output, in response to determining that the test coverage metric satisfies a coverage criterion and that the test convergence metric of the second test matrix satisfies a convergence criterion, to indicate that the test of the second analysis model is complete.

[0143] Clause 20. A computer-readable storage device pursuant to any of Clauses 15 to 19, wherein the instructions further cause the one or more processors to: generate the output in response to determining that the second error measure does not meet the error criterion, indicating that an error has been detected in the second analysis model, the second aircraft system, or both.

[0144] The examples described herein are intended to provide a general understanding of the structure of various implementations. These examples are not intended as a complete description of all components and features of devices and systems utilizing the structures or methods described herein. Many other implementations will be apparent to those skilled in the art upon review of this disclosure. Other implementations can be utilized and derived from this disclosure to allow for structural and logical substitutions and changes without departing from the scope of this disclosure. For example, method operations may be performed in an order different from that shown in the figures, or one or more method operations may be omitted. Therefore, this disclosure and the accompanying drawings are to be regarded as illustrative rather than restrictive.

[0145] Furthermore, although specific examples have been illustrated and described herein, it should be understood that any subsequent arrangement designed to achieve the same or similar results may replace the specific implementations shown. This disclosure is intended to cover any and all subsequent modifications or variations of the various implementations. When recalling this description, combinations of the above implementations, as well as other implementations not specifically described herein, will be apparent to those skilled in the art.

[0146] Submitting an abstract of the specification is conditional upon it not being used to interpret or limit the scope or meaning of the claims. Furthermore, regarding the foregoing “Detailed Description” content, for the purpose of streamlining this disclosure, various features may be grouped together or described in a single implementation. The examples above are illustrative and not limiting of this disclosure. It should also be understood that many modifications and variations can be made based on the principles of this disclosure. As reflected in the foregoing claims, the claimed subject matter may refer to less than all features of any of the disclosed examples. Therefore, the scope of this disclosure is defined by the foregoing claims and their equivalents.

Claims

1. A method for testing an aircraft system, the aircraft system comprising a first aircraft system and a second aircraft system, the method comprising the following steps: At the device, a first test matrix for testing the first aircraft system and a second test matrix for testing the second aircraft system are obtained, wherein the first test matrix describes a first operational test and the second test matrix describes a second operational test; During the first run test, sensor data is obtained at the device, the sensor data including first sensor data within the range specified by the first test matrix and second sensor data not specified by the first test matrix; At the device, the first operational test is evaluated by processing the first sensor data using a first analysis model of the first aircraft system, wherein the first analysis model is configured to generate first predictive sensor data based on the first sensor data. At the device, the first predictive sensor data is generated based on the evaluation of the first operational test; At the device, the second operational test is evaluated by processing the second sensor data using a second analysis model of the second aircraft system, wherein the second analysis model is configured to generate second predictive sensor data based on the second sensor data; At the device, the second predictive sensor data is generated based on the evaluation of the second operational test; At the device, a first error measure is generated by comparing a first subset of the sensor data with the first predictive sensor data; At the device, a second error measure is generated by comparing a second subset of the sensor data with the second predicted sensor data; At the device, a test coverage metric for the second test matrix is ​​determined, at least in part, based on the range of the second sensor data, wherein the test coverage metric indicates how much of the second test matrix has been covered; and The device provides an output to the display device, the output indicating the first error measure, the second error measure, and the test coverage measure.

2. The method according to claim 1, wherein, The first operational test includes at least one of ground testing, taxiing testing, and flight testing.

3. The method according to claim 1 or 2, wherein, The first analytical model includes physical models, empirical models, statistical models, simulation models, or combinations thereof.

4. The method according to claim 1 or 2, further comprising the following step: The second test matrix indicates the first data range of the first sensor; as well as In response to determining that the second data range from the first sensor has been processed using the second analysis model during the data collection period, the test coverage metric is determined at least in part based on a third data range, wherein the third data range is included in the first data range but not included in the second data range, and wherein the range of the second sensor data includes the second data range from the first sensor.

5. The method according to claim 1 or 2, further comprising the following step: The test convergence metric of the second test matrix is ​​determined at least in part based on the second error metric, wherein the output indicates the test convergence metric.

6. The method according to claim 1 or 2, further comprising the following step: A set of sensor data is received during the data collection period, wherein the set of sensor data includes the sensor data; The second test matrix is ​​evaluated based on the second analysis model and the sensor data set, wherein the multiple second run tests include the second run test; Multiple second error measures are determined based on the evaluation of the multiple second operational tests, wherein the multiple second error measures include the second error measures; and The test convergence metric of the second test matrix is ​​determined based on the plurality of second error measures.

7. The method according to claim 6, wherein, The test convergence metric is based, at least in part, on the standard deviation of the plurality of second error measures.

8. The method according to claim 1 or 2, further comprising the following step: In response to determining that the test coverage metric meets the coverage criterion and the test convergence metric of the second test matrix meets the convergence criterion, the output is generated to indicate that the test of the second analysis model is complete.

9. The method according to claim 1 or 2, further comprising the following step: In response to determining that the second error measure does not meet the error criteria, the output is generated to indicate that an error has been detected in the second analysis model, the second aircraft system, or both.

10. An aircraft, the aircraft comprising: An aircraft system, comprising a first aircraft system and a second aircraft system; The memory is configured to store multiple analysis models of the aircraft system, including a first analysis model of the first aircraft system and a second analysis model of the second aircraft system, wherein the first analysis model is configured to generate first predictive sensor data based on first sensor data, and the second analysis model is configured to generate second predictive sensor data based on second sensor data. A sensor configured to generate sensor data during a first run test of a first test matrix for testing the first aircraft system, wherein the sensor data includes first sensor data within a range specified by the first test matrix and second sensor data, wherein the second sensor data is not specified by the first test matrix, and wherein the first test matrix describes the first run test; One or more processors, said one or more processors being configured to: The first operational test is evaluated by processing the first sensor data using the first analysis model. The first predictive sensor data is generated based on the evaluation of the first run test; A second operational test is evaluated by processing the second sensor data using the second analysis model to test the second test matrix used to test the second aircraft system, wherein the second test matrix describes the second operational test; The second predictive sensor data is generated based on the evaluation of the second run test; A first error measure is generated by comparing a first subset of the sensor data with the first predictive sensor data. A second error measure is generated by comparing a second subset of the sensor data with the second predicted sensor data; and The test coverage metric of the second test matrix is ​​determined at least in part based on the range of the second sensor data, wherein the test coverage metric indicates how much of the second test matrix has been covered; and A display device configured to display an output indicating the first error measure, the second error measure, and the test coverage measure.

11. The aircraft according to claim 10, wherein, The sensors include airborne sensors, flight test sensors, or combinations thereof.

12. The aircraft according to claim 10 or 11, wherein, The second analytical model includes physical models, empirical models, statistical models, simulation models, or combinations thereof.

13. The aircraft according to claim 10 or 11, wherein, The one or more processors are further configured to generate the output in response to determining that the test coverage metric meets the coverage criterion and the test convergence metric of the second test matrix meets the convergence criterion, to indicate that the test of the second analysis model is complete.

14. The aircraft according to claim 10 or 11, wherein, The one or more processors are further configured to generate the output in response to determining that the second error measure does not meet the error criteria, indicating that an error has been detected in the second analysis model, the second aircraft system, or both.

15. A computer-readable storage device storing instructions that, when executed by one or more processors of the device, cause the one or more processors to: A first test matrix for testing the first aircraft system and a second test matrix for testing the second aircraft system are obtained, wherein... The first test matrix describes a first run test, and the second test matrix describes a second run test; During the first run test, sensor data is acquired, including first sensor data within the range specified by the first test matrix and second sensor data not specified by the first test matrix. The first operational test is evaluated by processing the first sensor data using a first analysis model of the first aircraft system, wherein the first analysis model is configured to generate first predictive sensor data based on the first sensor data. The first predictive sensor data is generated based on the evaluation of the first run test; The second operational test is evaluated by processing the second sensor data using a second analysis model of the second aircraft system, wherein the second analysis model is configured to generate second predictive sensor data based on the second sensor data; The second predictive sensor data is generated based on the evaluation of the second run test; A first error measure is generated by comparing a first subset of the sensor data with the first predictive sensor data. A second error measure is generated by comparing a second subset of the sensor data with the second predicted sensor data. The test coverage metric of the second test matrix is ​​determined at least in part based on the range of the second sensor data, wherein the test coverage metric indicates how much of the second test matrix has been covered; and An output is provided to a display device, the output indicating the first error measure, the second error measure, and the test coverage measure.

16. The computer-readable storage device according to claim 15, wherein, Each of the first and second operational tests includes at least one of ground tests, taxiing tests, and flight tests.

17. The computer-readable storage device according to claim 15 or 16, wherein, The instructions also cause the one or more processors to: The second test matrix is ​​determined to indicate the first data range of the first sensor; and In response to determining that the second data range from the first sensor has been processed using the second analysis model during the data collection period, the test coverage metric is determined at least in part based on a third data range, wherein the third data range is included in the first data range but not included in the second data range, and wherein the range of the second sensor data includes the second data range from the first sensor.

18. The computer-readable storage device according to claim 15 or 16, wherein, The instructions also cause the one or more processors to: A set of sensor data is obtained during the data collection period, wherein the set of sensor data includes the sensor data; The second test matrix is ​​evaluated based on the second analysis model and the sensor data set, wherein the multiple second run tests include the second run test; Multiple second error measures are determined based on the evaluation of the multiple second operational tests, wherein the multiple second error measures include the second error measures; and The test convergence metric of the second test matrix is ​​determined based on the plurality of second error measures.

19. The computer-readable storage device according to claim 15 or 16, wherein, The instructions also cause the one or more processors to generate the output, in response to determining that the test coverage metric meets the coverage criterion and the test convergence metric of the second test matrix meets the convergence criterion, to indicate that the test of the second analysis model is complete.

20. The computer-readable storage device according to claim 15 or 16, wherein, The instructions also cause the one or more processors to generate the output in response to determining that the second error measure does not meet the error criteria, indicating that an error has been detected in the second analysis model, the second aircraft system, or both.

Images