Providing confidence information associated with detected state transitions of dynamic system

Abstract

Providing confidence information associated with a detected state transition of a dynamic system is disclosed. A method includes obtaining, at a first device, a first output from a transition state model of a dynamic system. The first output indicates a first state of the dynamic system. The method includes obtaining, at the first device, a second output from the transition state model. The second output indicates a state transition from the first state to the second state. The method includes retrieving, at a first device, a transition confidence measure associated with a state transition from a first state to a second state from a transition confidence matrix. The method also includes providing, via the first device, a second state as an output in response to the transition confidence metric indicating that the state transition is valid.

Description

Technical Field

[0001] The present disclosure generally relates to providing confidence information associated with detected state transitions of a dynamic system. Background Art

[0002] Systems that change over time (e.g., natural systems and artificial systems) are dynamic systems. Some dynamic systems have a lot of complexity due to the characteristics of the processes of the dynamic system and due to the wide parameter space of the processes of the dynamic system. In order to simplify the modeling of dynamic systems, dynamic systems can be described by a collection of cluster states, each of which has a limited parameter space, and a separate model can be used for each state. For example, for the dynamic system of aircraft flight, the flight phases of initial climb, cruise, and descent can be used as states that form a simple description of aircraft flight, which has a parameter space including altitude and speed, and a simple model describing the behavior of the aircraft in each flight phase can be developed. Additional flight phases (e.g., taxi out, takeoff, route climb, route descent, approach, landing, taxi entry, etc.) can be defined as states that form a more complete and useful description of aircraft flight.

[0003] When states are used to describe a dynamic system, data of the dynamic system is usually analyzed to determine which state describes the dynamic system at a specific time. Unsupervised techniques can be used to infer the state of a dynamic system at a specific time. When using unsupervised techniques, the known state of the dynamic system for a specific condition is required to quantify the accuracy of the transition state model used to determine the state of the dynamic system. Characterizing the accuracy of the state of a dynamic system can be a laborious and complex task that often cannot be completed within the amount of time that the determined state is useful. A manual supervision process in which an experienced user manually processes a given set of states and compares them with the output of the transition state model can be used to quantify the accuracy of the transition state model, but the manual supervision process requires a lot of time to generate a target data set and depends on the size of the target data set. It is desirable to have an unsupervised technique to evaluate the accuracy of the transition between the states of a dynamic system, which can provide an indication that the determined state identified by the transition state model accurately reflects the current state of the dynamic system within a useful time frame. Summary of the invention

[0004] In a specific implementation, a system includes one or more processors and a memory. The memory contains instructions that can be executed by one or more processors. The instructions can be executed by one or more processors to obtain a first output from a transition state model. The first output indicates a first state of a dynamic system. The instructions can be executed by one or more processors to obtain a second output from the transition state model. The second output indicates a state transition from a first state to a second state. The instructions can be executed by one or more processors to retrieve a transition confidence metric associated with a state transition from a first state to a second state from a transition confidence matrix. The instructions can also be executed by one or more processors to provide a second state as an output when the transition confidence metric indicates that the state transition is valid.

[0005] In another specific implementation, a method includes obtaining a first output from a transition state model of a dynamic system at a first device. The first output indicates a first state of the dynamic system. The method includes obtaining a second output from the transition state model at the first device. The second output indicates a state transition from the first state to the second state. The method includes retrieving a transition confidence metric associated with the state transition from the first state to the second state from a transition confidence matrix at the first device. The method also includes providing the second state as an output via the first device in response to the transition confidence metric indicating that the state transition is valid.

[0006] In another specific implementation, a non-transitory computer-readable medium stores instructions that can be executed by one or more processors to obtain a first output from a transition state model of a dynamic system. The first output indicates a first state of the dynamic system. The instructions can be executed by one or more processors to obtain a second output from the transition state model. The second output indicates a state transition from the first state to the second state. The instructions can be executed by one or more processors to retrieve a transition confidence metric associated with the state transition from the first state to the second state from a transition confidence matrix. The instructions can also be executed by one or more processors to provide a second state as an output in response to the transition confidence metric indicating that the state transition is valid.

[0007] The features, functions, and advantages described herein can be achieved independently in various implementations or may be combined in yet other implementations further details of which can be found with reference to the following description and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 is a block diagram of a system for providing confidence information associated with state transitions of a dynamic system.

[0009] Figure 2 is a diagram depicting an example of a matrix for providing confidence information associated with state transitions of a particular dynamic system.

[0010] Figure 3is a diagram depicting an example of a transition matrix for a particular dynamic system indicating desired state transitions for the particular dynamic system.

[0011] Figure 4 Is a description based on Figure 2 A diagram of the first transformation confidence matrix of the matrix.

[0012] Figure 5 Is a description based on Figure 2 A diagram of a second transformation confidence matrix of the matrix.

[0013] Fig. 6A is a flow chart of a first part of a method of generating a transition confidence matrix that provides confidence information associated with detected state transitions of a dynamic system.

[0014] Figure 6B is a flow chart of a second part of a method of generating a transition confidence matrix that provides confidence information associated with detected state transitions of a dynamic system.

[0015] Figure 7 is a flow chart of a method of using a transition confidence matrix to provide confidence information associated with a detected state transition of a dynamic system.

[0016] Figure 8 is a block diagram of a computing environment including a computing device configured to provide confidence information associated with a detected state transition of a dynamic system. DETAILED DESCRIPTION

[0017] Aspects disclosed herein present systems and methods for providing confidence information associated with detected state transitions of a dynamic system. A dynamic system may be any dynamic system that changes over time, is characterized by a plurality of states, and is associated with a transition state model. For example, the dynamic system may correspond to an aircraft flight, a turbulent fluid flow system, a batch chemical reaction system, etc. In the present disclosure, aircraft flight is primarily used as the dynamic system in these examples, but the description is also applicable to other dynamic systems.

[0018] A transition state model of a dynamic system is used by a device (e.g., a computer system) to determine a state of the dynamic system based on data associated with the dynamic system and to provide an output that can be used by the device or another device. For example, a transition state model of an aircraft flight generates an output based on Automatic Dependent Surveillance-Broadcast (ADS-B) data to indicate the state of each aircraft flight in the aircraft flight at a particular time. In an illustrative example, the states include taxi-out, takeoff, initial climb, en-route climb, cruise, en-route descent, descent, approach, landing, and taxi-in. The state output by the transition state model is determined independently of a previous output state determined by the transition state model.

[0019] When the transition state model is reliable (e.g., determined to be sufficiently accurate to be relied upon to provide meaningful output), the output generated by the transition state model can be used by the device or another device to provide information about the flight of an aircraft to the public, can be used by air traffic control systems and personnel to monitor and control aircraft in an airspace region associated with an airport, etc. The meaningful output corresponds to the actual state of the aircraft, without the majority of the output corresponding to states that are impossible based on previous states. For example, if the output of the transition state model of the flight of an aircraft indicates that the state is taxi-in, the output is meaningful if the previous state is landing, and the output is meaningless if the previous state is a state other than landing. If the previous state is landing, the output can be used to provide an estimate of the arrival of the aircraft at the gate. Conversely, if the previous state is not landing, using the "taxi-in" output to provide an estimate of the arrival of the aircraft at the gate will provide an unrealistic arrival time and can cause one or more human problems that rely on the estimate.

[0020] Providing a confidence metric associated with the output of the transition state model makes the output of the transition state model more useful. The confidence metric is provided as a transition confidence matrix of the effectiveness of the transition from the first state to the second state indicated by the transition state model. The confidence metric does not address the possibility of a specific transition identified by the transition state model occurring, but provides information about whether a specific transition is a valid transition of the dynamic system. For example, for some dynamic systems, a specific transition from a first specific state is unlikely to occur (e.g., 0.1% or less chance of occurrence), but such transitions are known to occur occasionally. The entry for a specific transition in the transition confidence matrix is ​​a non-zero value (i.e., a value corresponding to a percentage or logical value 1) to indicate that a specific transition can occur and the occurrence of this transition can be valid. Thus, an identified state transition that is unlikely to occur but is still possible will not be automatically identified as an invalid transition because it is unlikely to occur.

[0021] The transition confidence matrix is ​​generated based on the analysis of historical data of the dynamic system within a first specific time period, and the matrix can be updated at specific time intervals (e.g., daily, weekly, monthly, seasonally, or some other time intervals). The first specific time period corresponds to a useful time period associated with a specific dynamic system (e.g., one or more days, one or more weeks, one or more months, one or more years, seasons based on data from one or more previous years, or some other time period). The transition confidence matrix may include columns representing the current state of the dynamic system and rows representing the previous state of the dynamic system. In some implementations, the entries in the transition confidence matrix correspond to the percentage (or fraction) of the occurrence of the transition from the state indicated by the row to the state indicated by the column, and when the matrix presents percentages, the sum of the values ​​in the row is 100 or close to 100 due to rounding errors.

[0022] In some implementations, the transition confidence matrix is ​​modified to set statistically insignificant entries (e.g., entries having values ​​corresponding to less than 1%, 0.1%, 0.01%, or some other threshold) to zero. In addition, some entries of the matrix associated with transitions known not to have occurred are set to zero. For example, for an aircraft flight, a row of the matrix may correspond to a taxi entry state. The taxi entry state is the last state for each flight, and there are no transitions to another state during the taxi entry. When a matrix based on historical data includes an entry for a taxi entry state that corresponds to a transition to another state, such an entry may be set to zero because such a transition will not occur.

[0023] In some implementations, the transition confidence matrix is ​​used with entries presented as percentages or fractions (e.g., numerical values). In some implementations, the transition confidence matrix is ​​transformed into a transition confidence matrix of 1s and 0s (e.g., logical values) to simplify the transition confidence matrix. In an example, entries of the matrix that are zero remain zero, entries with values ​​less than a threshold are set to zero, and the remaining entries are assigned a value of 1. According to some aspects, entries that are known to have a likelihood of occurring (even if the calculated value of the entry based on historical data is below the threshold) are also assigned a value of 1.

[0024] In some implementations, the device uses a transition confidence monitor to determine the confidence of a transition indicated by a transition state model using a transition confidence matrix. The transition confidence monitor uses the transition confidence matrix to evaluate the validity of a transition to a second state indicated by the transition state model and generates an output. In an example, when the confidence metric from the transition confidence matrix is ​​a non-zero value, the output of the transition confidence monitor includes a second state. The device or another device may use the second state (e.g., the current state of the dynamic system) as information about the dynamic system or as a parameter for determining other information about the dynamic system. In another example, when the confidence metric from the transition confidence matrix for a transition indicated by the transition state model is zero, the transition confidence monitor does not output the second state. In some implementations, when the confidence metric indicates that the state transition is invalid, the transition confidence monitor monitors the passage of time after the state transition, and if the transition confidence monitor does not receive input from the transition state model indicating another state transition before exceeding a threshold time (e.g., 1 second, 5 seconds, 10 seconds, or some other time span), the transition confidence monitor provides a second state as an output.

[0025] The output of the second state of the transition confidence monitor can be used by the device or another device using the output from the transition confidence monitor to determine how to utilize the information of the transition of the dynamic system to the second state. In some examples, the output can be used by a control system to control an instance of the dynamic system, by a notification system that provides information indicating the state of the dynamic system, by other types of systems that use the state of the dynamic system as input, or a combination thereof.

[0026] The confidence measures of the transition confidence matrix can also be used as a basis for revising the transition state model. For example, for an aircraft flight system, there should be little or no transition from cruise to initial climb, such as Figure 3 If a confidence metric for the transition from cruise to initial climb based on an analysis of historical data of the aircraft's flight indicates that the transition from cruise to initial climb occurs somewhat frequently (e.g., occurs within 5% of the time of all transitions from cruise to another state), information may be provided to an entity associated with a transition state model to request modification of parameters used by the transition state model to determine that the aircraft state is in the initial climb state, the cruise state, or both, to reduce or eliminate occurrences indicating transitions from cruise to initial climb.

[0027] Most dynamic systems that are monitored generate a large amount of data. The amount of historical data that needs to be analyzed to achieve quantification of confidence metrics makes it impractical for humans to determine such confidence metrics. A technical advantage of the present disclosure is that when the state transitions of a dynamic system can be determined by a transition state model of the dynamic system, one or more computer systems can analyze the historical data of the dynamic system to generate a confidence metric matrix for the dynamic system. The matrix can be used by the computer system or by other computer systems that receive the matrix to provide an output associated with the confidence of the transitions detected by the transition state model.

[0028] The accompanying drawings and the following description illustrate specific exemplary implementations. It should be understood that those skilled in the art will be able to design various arrangements that, although not explicitly described or shown herein, embody the principles described herein and are included within the scope of the claims following this description. In addition, any examples described herein are intended to aid in understanding the principles of the present disclosure and are to be interpreted as not limiting. Therefore, the present disclosure is not limited to the specific embodiments or examples described below, but is defined by the claims and their equivalents.

[0029] DETAILED DESCRIPTION OF THE EMBODIMENTS The present invention is described herein with reference to the accompanying drawings. In the description, common features are denoted by common reference numerals throughout the drawings.

[0030] As used herein, various terms are used only for the purpose of describing particular implementations and are not intended to be limiting. For example, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. Further, some features described herein are singular in some implementations and plural in other implementations. For illustration, Figure 1 Describes a system comprising one or more processors ( Figure 1 The first computer system 108 (the "processor" 120 in the figure) is referred to as the first computer system 108, which means that in some embodiments the first computer system 108 includes a single processor 120 and in other embodiments, the first computer system 108 includes multiple processors 120. For ease of reference herein, such features are generally introduced as "one or more" features and are then referred to in the singular or optional plural (indicated by "(s)") unless aspects related to multiple features are described.

[0031] The term "phase" is used interchangeably with the term "state". The terms "comprise", "comprises" and "comprising" are used interchangeably with "include", "includes" or "including". In addition, the term "wherein" is used interchangeably with the term "wherein". As used herein, "exemplary" indicates an example, implementation and / or aspect, and should not be interpreted as limiting or indicating a preference or preferred implementation. As used herein, ordinal terms (e.g., "first", "second", "third", etc.) used to modify elements (such as structures, components, operations, etc.) do not themselves indicate any priority or order of the element relative to another element, but only distinguish the element from another element with the same name (but for the use of ordinal terms). As used herein, the term "set" refers to a grouping of one or more elements, and the term "plurality" refers to a plurality of elements. As used herein, A "and / or" B may mean "A and B", or "A or B", or both "A and B" and "A or B" are applicable or acceptable.

[0032] As used herein, "generate", "calculate", "use", "select", "access" and "determine" are interchangeable unless the context indicates otherwise. For example, "generate", "calculate" or "determine" a parameter (or signal) may refer to actively generating, calculating or determining a parameter (or signal) or may refer to using, selecting or accessing an already generated parameter (or signal), such as by another component or device. As used herein, "coupling" may include "communicative coupling", "electrical coupling" or "physical coupling", and may also (or alternatively) include any combination thereof. Two devices (or components) may be directly or indirectly coupled (e.g., communicatively coupled, electrically coupled or physically coupled) via one or more other devices, components, wires, buses, networks (e.g., wired networks, wireless networks, or combinations thereof), etc. Two electrically coupled devices (or components) may be included in the same device or in different devices, and may be connected via electronic devices, one or more connectors, or inductive coupling, as illustrative, non-limiting examples. In some embodiments, two devices (or components) that are communicatively coupled (e.g., in electrical communication) may directly or indirectly send and receive electrical signals (digital or analog), for example, via one or more wires, buses, networks, etc. As used herein, "directly coupled" is used to describe two devices that are coupled (e.g., communicatively coupled, electrically coupled, or physically coupled) without intervening components.

[0033] Figure 1A block diagram of a system 100 configured to provide confidence information associated with a state transition of a dynamic system 102 is depicted. The state transition is determined by a transition state model 104. The dynamic system 102 may be any dynamic system 102, wherein the transition state model 104 for the dynamic system 102 detects transitions between states of the dynamic system 102 based on data associated with the dynamic system 102. For example, in some implementations, the dynamic system 102 includes an aircraft flight and the transition state model 104 uses a flight phase processor developed by Aviation Insight, Inc. (Chevrolet Chase, Maryland). The flight phase processor uses fuzzy logic methods to infer the flight phase of a given flight in real time based on Automatic Dependent Surveillance-Broadcast (ADS-B) data. The flight phase is based on a particular cluster of interest. In a specific implementation, the flight phase includes taxi out, takeoff, initial climb, route climb, cruise, route descent, descent, approach, landing, and taxi approach. In other implementations, the flight phase may include fewer flight phases, or one or more additional flight phases.

[0034] The system 100 includes a dynamic system 102; a database 106 storing data associated with the dynamic system 102; one or more first computer systems 108, the one or more first computer systems 108 configured to determine a transition confidence matrix 110 based on an analysis of a historical set of transition data, wherein the historical set of transition data is determined using a transition state model 104 based on the set of historical data from the database 106; and one or more second computer systems 112 configured to use a transition confidence monitor 114 to verify transitions between states determined by the transition state model 104 based on an analysis of real-time data or near real-time data, the real-time data or near real-time data is from the database 106 or from one or more instances of the dynamic system 102. The second computer system 112 is configured to output data based on detected transition confidence metric values ​​associated with detected transitions. The second computer system 112 may include or provide output to one or more control systems 116 of the dynamic system 102, one or more notification systems 118 associated with the dynamic system 102, other types of devices, or combinations thereof.

[0035] exist Figure 1 , the transition state model 104 is depicted as part of the first computer system 108 and the second computer system 112. In some implementations, the transition state model 104 is part of a separate device that provides output to the first computer system 108, the second computer system 112, or both. Figure 11, first computer system 108, second computer system 112, and notification system 118 are described as separate components, but in some embodiments, the functionality of two or more of these components may be combined. Likewise, the functionality of control system 116 depicted in second computer system 112 may be implemented in a separate device. For example, first computer system 108 may include the functionality of second computer system 112 and provide output to separate control system 116, to notification system 118, or both; second computer system 112 may include the functionality of notification system 118 and provide control signals to an instance of dynamic system 102 (e.g., a specific batch reactor of the dynamic system corresponding to a batch reactor for producing a specific product) based on data received from database 106, data from sensors associated with the instance of dynamic system 102, or both; and so on.

[0036] The database 106 receives data from the dynamic system 102 and stores the data. The data includes specific data that the transition state model 104 can use to determine when the dynamic system 102 transitions from one state to another state. In an example where the dynamic system 102 corresponds to an aircraft flight, the data stored in the database 106 is ADS-B data including time information, flight information, speed information, position information, other information, or a combination thereof. The first computer system 108, the second computer system 112, or both use the transition state model 104 to process the data from the database 106 to identify a first specific flight and a first flight phase associated with the first specific flight at a specific time, store the information associated with the first specific flight as stored information, and determine whether the flight phase has changed based on a comparison of the first flight phase with the flight phase determined from the next entry for the flight in the stored information. When the comparison indicates that the flight phase has changed, the transition state model 104 outputs specific information including an identifier of the first specific flight, a specific time, and the first flight phase.

[0037] The first computer system 108 is configured to generate a transition confidence matrix 110. The first computer system 108 includes one or more processors 120 and a memory 122. The memory includes data 124 and instructions 126. The instructions 126 are executable by the processor 120 to perform one or more operations. The instructions 126 include specific instructions (e.g., scripts, programs, subroutines, applications, etc.) that can be executed by the processor 120 and include a transition matrix generator 128 and a transition state model 104.

[0038] The first computer system 108 is configured to execute the transition matrix generator 128 using the processor 120 to generate the transition confidence matrix 110 upon user request or at specific time intervals. During use of the first computer system 108 to generate the transition confidence matrix 110, the first computer system 108 accesses the database 106 to retrieve a collection of historical data for the dynamic system 102. The transition matrix generator 128 causes the collection of historical data to be processed using the transition state model 104. Transition data for each transition detected by the transition state model 104 in the collection of historical data is stored in a first data set 130 (e.g., a data frame) in the data 124 of the memory 122.

[0039] In some implementations, the second data set 132 is generated based on the first data set 130. The second data set includes at least one entry from the first data set 130 for each individual case identified in the collection of historical data, and includes one entry for each type of state for each case. For example, if the collection of historical data identifies a first case of the dynamic system 102 associated with 14 individual states, and data from the transition state model 104 indicates that there are 1653 state changes for the first case associated with 11 individual types of states; there will be entries corresponding to the 1653 state transitions in the first data set, and for this case, the entries in the second data set 132 will include 11 entries, which will correspond to one entry for each of the 11 individual state types.

[0040] After generating the second data set 132, the content of the second data set 132 is analyzed to identify a first case in which a realistic representation of a possible state is found based on one or more criteria. The one or more criteria may include the presence of a start state and an end state, and may include one or more additional states. For example, when the dynamic system 102 is an aircraft flight, each flight corresponds to a case, and for realistic flights, the second data set 132 needs to include entries corresponding to the start state of taxi-out and the final state of taxi-in, and these entries should also include takeoff, initial climb, approach, and landing. When the entries in the second data set for a particular case (i.e., a particular flight) do not include each of the states of taxi-out, takeoff, initial climb, approach, landing, and taxi-in, the particular case is not identified as one of the first cases.

[0041] The first data set 130 is internally combined with the second data set 132, and the second data set 132 is configured to modify the first data set 130 to include only the first cases that meet one or more criteria (e.g., cases with a minimum number of realistic states). The modified first data set 130 is processed to generate a first matrix. The entries in the first matrix correspond to the confidence measures of the indicated transitions. According to one aspect, the first matrix is ​​a square array with rows and columns. The rows and columns correspond to an ordered list of states of the dynamic system 102, wherein the columns indicate the states detected by the transition state model 104, and the rows indicate the previous states to the states indicated in the columns. Processing the modified first data set 130 includes identifying the states indicated in the first data set as final states and using a data processing function (e.g., sql.lag function) to determine the previous states associated with each final state. Each state in the previous state is identified as an initial state.

[0042] The first data set 130 is processed to determine the first sum of occurrences of the first case in which the state is identified as an instance of the initial state for each type of state. The first data set is further processed to determine the second sum of the first case in which the transition from the initial state to the final state occurs for each initial state and final state pairing. For each initial state and final state pairing, the entries of the first matrix are generated. The entries of a specific initial state and a specific final state are based on the second sum of the specific initial state and the specific final state divided by the first sum associated with the specific initial state. The division produces a fraction, and the entries in a specific row add up to one (or close to one due to rounding errors). The fraction can be converted into a percentage by multiplying the fraction by 100.

[0043] The information in the first matrix may be compared with information based on user experience associated with the dynamic system 102, based on expected transitions of the dynamic system 102, or both. The expected transitions may be presented as an expected transition matrix, may be individual values ​​for a particular transition from a first state to a second state, or a combination thereof. The comparison may be used to determine whether the first matrix (i.e., including actual values ​​for most or all entries) is available. When it is determined that the first matrix is ​​available, the first matrix may be used as the transition confidence matrix 110, or the first matrix may be modified to generate the transition confidence matrix 110. When it is determined that the first matrix is ​​not available or a better representation can be generated, the first matrix or specific entries in the first matrix may be used as a basis for updating the transition state model 104, or updating the transition state model 104 to more accurately indicate the transition of the dynamic system 102. For example, as discussed above, information may be provided to an entity associated with the transition state model 104 to request that the parameters (e.g., altitude threshold and speed threshold) used by the transition state model 104 to determine that the aircraft state is in the initial climb state be modified so as to reduce or eliminate the occurrence of indicating a transition from cruise to initial climb. If an updated transition state model 104 is provided and put into use, the transition confidence matrix 110 is updated so that the transition confidence matrix 110 is based on the transition state model 104 used to detect state transitions.

[0044] Figure 2 A first matrix 200 is depicted for a flight system generated by the first computer system 108 for a particular time frame based on historical data from the database 106, wherein entries are expressed as percentages. Entries with values ​​exceeding 20% ​​are depicted in bold, entries with values ​​from 10% to 20% are shown in italics, and entries with values ​​less than 10% are shown in normal text to facilitate visualization of the information presented. Figure 2-Figure 4 The row headings (ie, previous state), column headings (ie, current state), and state indicators 202 - 222 presented in association with the matrices 200 , 300 , and 110 are similarly provided to facilitate visualization of the presented information and are not part of the matrices 200 , 300 , and 110 .

[0045] The number of flights that meet one or more criteria for generating the modified first data set 130 exceeds 175,000 flights, and the historical data used to generate the first matrix 200 is based on ADS-B data from May 2022 to July 2022. The first matrix 200 includes a taxi-out (TxO) state 202, a takeoff (TkOff) state 204, an initial climb state (iC) 206, a course climb (state eC) 208, a cruise state (Cr) 210, a course descent state (eD) 212, a descent state (D) 214, an approach state (A) 216, a landing state (L) 218, a taxi-in state (TxI) 220, and an unknown state (Unk) 222. The unknown entries 222 correspond to times when the ADS-B data from the database 106 is missing or significantly erroneous (e.g., the position data indicates that the aircraft is above the earth). ADS-B data for a flight may be lost due to the position of the aircraft being outside the range of a detector monitoring ADS-B data (e.g., the aircraft is located over the ocean hundreds of miles from the nearest fixed-position ADS-B detector providing data to database 106), due to incomplete reception of ADS-B messages by the detector, other reasons, or a combination thereof. ADS-B data from database 106 may clearly be erroneous due to interference, corruption, or other reasons.

[0046] Figure 3 Describes the use Figure 1 A transition matrix 300 of a flight system of the aircraft is shown. The shaded entries depict transitions that are expected to occur during the flight of the aircraft based on the user experience. Figure 2 The matrix 200 and Figure 3 A comparison of the transition matrix 300 of FIG. 200 shows that the matrix 200 agrees fairly well with the transition matrix 300. The comparison shows that the matrix 200 and the transition matrix 300 predict the same behavior for taxi-out, takeoff, cruise, approach, and landing, and have only minor differences for initial climb, route climb, and route descent. Since the taxi-in state is the last state of the flight, no taxi-in state changes to any other state occur. Although the data presented in the second to last row of the transition confidence matrix 200 are problematic due to the relatively high percentage values ​​of transitions that should not occur, this is because these values ​​represent the percentage of the total number of times a transition from the taxi-in state to another state is detected. In the data used to generate these values, the total number of occurrences of transitions from the taxi-in state to another state is low (i.e., approximately 3000 occurrences), which is a small number compared to the large total number of flights used to generate the transition confidence matrix (i.e., more than 175,000). Thus, the number of occurrences of transitions from the taxi-in state to another state may be statistically insignificant.

[0047] In some implementations, the matrix 200 is used as the transition confidence matrix 110. In other embodiments, the matrix 200 is modified to a second matrix, and the second matrix is ​​used as the transition confidence matrix 110. The modification may be based on known limitations of the dynamic system 102 (e.g., for the dynamic system 102 of the aircraft flight, the taxi entry state 220 does not change to another state), based on zeroing out statistically insignificant entries of the matrix 200 (e.g., corresponding to entries less than 1% or some other probability of occurrence), based on other considerations, or a combination thereof. In some implementations, the matrix 200 or the second matrix may be modified to a matrix in which entries that are not zero or greater than a confidence threshold (e.g., 1%, 0.5%, or some other value) are 1 and entries that are zero or at or below the confidence threshold are 0. The resulting matrix is ​​used as the transition confidence matrix 110. Figure 4 A first implementation of a transition confidence matrix 110 generated by modifying the matrix 200 to have zero values ​​for the taxi entry row and to have zero values ​​for less than 1% of the entries in the matrix 200 is depicted. Figure 5 Describes the Figure 4 A second implementation of the transition confidence matrix 110 is generated by converting the transition confidence matrix 110 into a matrix having entries with logical values ​​0 or 1.

[0048] return Figure 1 , the transition confidence matrix 110 generated by the first computer system 108 (e.g., Figure 4 The transition confidence matrix 110 or Figure 5 The transition confidence matrix 110 (as an illustrative example) can be provided to the second computer system 112. The second computer system 112 uses the transition confidence matrix 110 to determine whether the transition detected by the transition state model is valid based on real-time data or near real-time data from the database 106 or sensors monitoring the instance of the dynamic system 102.

[0049] The second computer system 112 includes one or more processors 134 and memory 136. The memory 136 includes data 138 and instructions 140. The instructions 140 are executable by the processor 134 to perform one or more operations. The instructions 140 include specific instructions (e.g., scripts, programs, subroutines, applications, etc.) that can be executed by the processor 134. Figure 1 The specific instructions depicted in include a transition confidence monitor 114 , a transition state model 104 , and a control system 116 .

[0050] The first computer system 108 is configured to execute the transition confidence monitor 114 using the processor 134 to generate an output indicating a particular state when the transition confidence monitor 114 determines that a transition to the particular state is valid. When the transition confidence monitor 114 is activated, the transition confidence monitor 114 initiates the use of the transition state model 104 to determine a state transition based on real-time data or near real-time data from the database 106 or from sensors monitoring an instance of the dynamic system 102. When the transition state model 104 provides input indicating a transition of the dynamic system 102 for a case, the transition confidence monitor 114 retrieves a previous state for the case (which is set to "unknown" when the input for the case is first received) and retrieves a confidence measure for the transition by accessing entries in the transition confidence matrix 110 corresponding to the previous state and the current state as indicated by the input.

[0051] When the confidence value is not zero, a valid transition from the previous state to the state provided by the transition state monitor 104 is indicated. The transition confidence monitor 114 saves the state indicated by the transition as the previous state and generates output data containing the state indicated by the transition. The output data is provided as input to another device (e.g., notification system 118), as input for other instructions being executed by the processor 134 (e.g., control system 116), or both.

[0052] When the confidence value is zero, the transition to the state provided by the transition state monitor 104 is an invalid transition, and the transition confidence monitor 114 does not provide output indicating the state provided by the transition state monitor 104. In some implementations, when the confidence value is zero, the transition confidence monitor 114 waits for additional output from the transition state model 104 without providing output data. When additional output is not received from the transition state model 104 before a threshold amount of time has passed, the state indicated by the transition is sent as output data from the transition confidence monitor 114.

[0053] The output of the transition confidence monitor 114 may be provided to a notification system 118. In some implementations, the notification system 118 generates updated information associated with an instance of the dynamic system based on receiving the output from the transition confidence monitor 114 and provides the updated information to one or more display devices. For example, the notification system 118 may receive an output from the transition confidence monitor 114 indicating a transition to a taxi-in state for an aircraft associated with a particular flight at an airport. Based on the indication of the transition to the taxi-in phase, the notification system 118 may access the average time for the aircraft to taxi-in after landing at the airport and calculate an estimate of the aircraft's stand (e.g., gate) arrival time. The notification system 118 may create updated information for the flight and present the updated information as part of the information presented via the airport arrival / departure monitor.

[0054] The control system 116 includes a model of the state of the dynamic system 102 (e.g., instructions executable by the processor 134 to simulate the state of the dynamic system). In response to receiving the output from the transition confidence monitor 114, the control system 116 uses the state indicated in the output as the current state of the dynamic system 102. The control system 116 is configured to simulate the current state using the model corresponding to the current state and generate one or more control signals to send to one or more controllers of the dynamic system 102 to affect one or more conditions of the instance of the dynamic system 102 (e.g., flight near a particular airport).

[0055] Fig. 6A 1 is a flow chart of a first portion of a method 600 for generating a transition confidence matrix 110 for a dynamic system 102. In an embodiment, the method 600 is composed of Figure 1 The method 600 includes obtaining historical data associated with a state change of the dynamic system 102 from the database 106 at block 602 .

[0056] The method 600 includes processing the historical data using the transition state model 104 at block 604. The method 600 includes generating a first data set 130 of transitions detected by the transition state model 104 at block 606. For each state transition detected by the transition state model 104 for the historical data, the first data set 130 includes: an identifier of a case of the dynamic system 102 undergoing the state transition, a time of the state transition, and a final state associated with the state transition. The method 600 includes generating a second data set 132 at block 608, the second data set 132 including a single entry for each case of each different type of final state identified.

[0057] The method 600 includes, at block 610, processing the second data set 132 to identify a first case that satisfies one or more criteria. The one or more criteria include the presence of one or more specific states in the second data set 132 for the case. For example, for some dynamic systems 102, the one or more criteria require that the state in the second data set 132 for a specific case include a first state (e.g., landing for a dynamic system 102 for aircraft flight). Additional criteria in the one or more criteria may require that the state in the second data set 132 for a specific case include one or more additional states (e.g., takeoff and taxi approach for a dynamic system 102 for aircraft flight).

[0058] The method 600 includes processing the first data set 130 to identify, for each state transition of the first case, an initial state preceding the final state based on the time of the state transition at block 612. The method 600 includes determining, for each type of state, a first sum of first instance cases in the first data set 130, where the state is identified as an initial state at block 614. The method also includes determining, for each initial state and final state pair, a second sum of occurrences of transitions from the initial state to the final state for the first case in the first data set 130 at block 616.

[0059] The method 600 includes generating entries of the transition confidence matrix 110 at block 618. The entry associated with the particular initial state and the particular final state is based on the second sum of the particular initial state and the particular final state divided by the first sum associated with the particular initial state. In some implementations, the method 600 ends after block 618. In other implementations, the method may optionally be as follows: Figure 6B Continue as described in .

[0060] Figure 6B Flowchart of a second portion of the method 600 for generating a transition confidence matrix 110 for a dynamic system 102. In some embodiments, the method 600 begins with Fig. 6A The method 600 continues at block 618 of the present invention. The method 600 includes determining at decision block 620 whether to compare one or more confidence metrics of the transition confidence matrix 110 with one or more expected confidence metrics. When the determination at decision block 620 is to compare one or more confidence metrics, the method 600 includes comparing one or more confidence metrics of the transition confidence matrix 110 with one or more expected confidence metrics at block 622. The one or more expected confidence metrics are based on the user's experience with the dynamic system 102 and include an indication of a state transition that is expected to occur or is known to occur, an indication of a state transition that is expected not to occur or is known not to occur, or a combination thereof. The method 600 continues at decision block 624. When the determination at decision block 620 is not to compare one or more confidence metrics, the method 600 continues at decision block 630.

[0061] The method 600 includes determining, at a decision block 624, whether the transition confidence matrix 110 is acceptable. In some implementations, the determination is based on the confidence metrics being consistent or substantially consistent with expected confidence metrics without having a large number of confidence metrics that would incorrectly indicate invalid state transitions. When the determination at the decision block 624 is that the transition confidence matrix 110 is not acceptable, the method 600 includes providing information to an entity associated with the transition state model 104 at a block 626 to enable generation of the transition state model 104 that better identifies state transitions of the dynamic system 102. The method 600 ends at a block 628.

[0062] When the determination at decision box 624 is that the transition confidence matrix 110 is acceptable or the determination at decision box 620 is not to compare one or more confidence metrics to one or more expected values, the method 600 includes determining whether to modify the transition confidence matrix 110 at decision box 630. The determination may be based on the following desires: to change one or more transition confidence metrics that are known to be incorrect to a correct value; to change values ​​less than a threshold to zero; to transform the transition confidence matrix 110 to a matrix that includes only ones and zeros; or a combination thereof. When the determination at decision box 630 is not to modify the transition confidence matrix 110, the method 600 ends at box 628. When the determination at decision box 630 is to modify the transition confidence matrix 110, the method 600 includes adjusting one or more of the confidence metrics of the transition confidence matrix 110 at box 632. The method 600 then ends at box 628.

[0063] Figure 7 is a flow chart of a method 700 for providing confidence information associated with a detected state transition of a dynamic system using a transition confidence matrix 110. In some implementations, the method 700 comprises Figure 1 The second computer system 112 of the present invention is executed using the transition confidence monitor 114. The method 700 includes obtaining a first input from a transition state model 104 of the dynamic system 102 at block 702. The first input indicates a first state of the dynamic system 102. The method 700 also includes obtaining a second input from the transition state model 104 at block 704. The second input indicates a state transition from the first state to the second state.

[0064] The method 700 includes retrieving a transition confidence metric associated with the state transition from the first state to the second state from the transition confidence matrix 110 at block 706. In some implementations, the transition confidence matrix 110 is downloaded to the second computer system 112. In some implementations, the second computer system 112 is configured to generate the transition confidence matrix 110.

[0065] The method 700 includes providing a second state as an output in response to the transition confidence metric indicating that the state transition is valid, at block 708. A confidence metric greater than zero in the transition confidence matrix 110 indicates a valid transition.

[0066] The method 700 also includes waiting for additional input from the transition state model 104 without providing output when the transition confidence metric indicates that the state transition is invalid at block 710. A confidence metric of zero in the transition confidence matrix 110 indicates an invalid transition.

[0067] Figure 8 is an illustration of a block diagram of a computing environment 800 including a computing device 802 configured to support implementation of computer-implemented methods and computer-executable program instructions (or code) according to the present disclosure. For example, the computing device 802 or a portion thereof may execute instructions to perform or cause the device to perform the method described in the embodiments of the present disclosure. Figure 1-Figure 7 In one embodiment, the computing device 802 is Figure 1 The dynamic system 102, the database 106, the first computer system 108, the second computer system 112 and the notification system 118 are or Figure 1 Components of the dynamic system 102, the database 106, the first computer system 108, the second computer system 112 and the notification system 118.

[0068] The computing device 802 includes one or more processors 804. The processor 804 communicates with a system memory 806, one or more storage devices 808, one or more input / output interfaces 810, one or more communication interfaces 812, or a combination thereof. The system memory 806 includes non-transitory computer-readable media, including volatile storage devices (e.g., random access memory (RAM) devices), non-volatile storage devices (e.g., read-only memory (ROM) devices, programmable read-only memory, and flash memory), or both. The system memory 806 includes an operating system 814, which may include a basic input / output system for starting the computing device 802 and a complete operating system that enables the computing device 802 to interact with users, other programs, and other devices. The system memory 806 includes one or more applications 816 (e.g., instructions) that can be executed by the processor 804. For example, when the computing device 802 is the first computer system 108, the one or more applications 816 include the transition state model 104 and the transition matrix generator 128; and when the computing device 802 is the second computer system 112, the one or more applications 816 include the transition state model 104, the transition confidence monitor 114, and the control system 116.

[0069] In some configurations, the processor 804 communicates with one or more storage devices 808. For example, the storage device 808 includes a non-transitory computer-readable medium, which may include a non-volatile storage device, such as a magnetic disk, an optical disk, or a flash memory device. The storage device 808 may include removable and non-removable storage devices. The storage device 808 may be configured to store an operating system, an image of the operating system, applications, and program data. In a specific embodiment, the system memory 806, the storage device 808, or both include a tangible computer-readable medium that is incorporated in hardware and is not a signal.

[0070] In some configurations, the processor 804 communicates with one or more input / output interfaces 810 that enable the computing device 802 to communicate with one or more input / output devices 818 to facilitate user interaction. The input / output interface 810 may include a serial interface (e.g., a universal serial bus (USB) interface or an Institute of Electrical and Electronics Engineers (IEEE) interface), a parallel interface, a display adapter, an audio adapter, and other interfaces ("IEEE" is a registered trademark of the Institute of Electrical and Electronics Engineers of Piscataway, New Jersey). The input / output device 818 may include a keyboard, a pointing device, a display (e.g., one or more monitors, one or more meters, etc.), a speaker, a microphone, a touch screen, a rotatable selector, a joystick, a knob, a slider, a switch, and other devices. The processor 804 detects an interaction event based on user input received via the input / output interface 810. In addition, the processor 804 sends a display to a display device via the input / output interface 810.

[0071] In some configurations, the processor 804 may communicate with one or more devices 820 via one or more communication interfaces 812. The one or more devices 820 may include external computing devices that communicate via a communication network, as well as controllers, sensors, and other devices that are connected to the computing device 802 via a wired or wireless local connection. For example, when the computing device 802 is the second computer system 112, the computing device 802 is configured to communicate with the database 106, the dynamic system 102, the notification system 118, or a combination thereof via the interface 812. The one or more communication interfaces 812 may include a wired Ethernet interface, an IEEE 802 wireless interface, other wireless communication interfaces, one or more converters that convert analog signals to digital signals, one or more converters that convert electrical signals to optical signals, one or more converters that convert received optical signals to electrical signals, or other network interfaces.

[0072] In some embodiments, a non-transitory computer-readable medium stores instructions that, when executed by one or more processors, cause the one or more processors to initiate, execute, or control operations to perform some or all of the functionality described above. For example, the instructions may be executed to implement Figure 1-Figure 7 In some embodiments, the above-mentioned operations or methods may be implemented by one or more processors (e.g., one or more central processing units (CPUs), one or more graphics processing units (GPUs), one or more digital signal processors (DSPs)) executing instructions, by dedicated hardware circuits, or any combination thereof. Figure 1-Figure 7 Part or all of one or more associated operations or methods.

[0073] Certain aspects of the present disclosure are described in the following set of related examples:

[0074] According to Example 1, a system includes: one or more processors; and a memory including instructions executable by the one or more processors to perform the following operations: obtain a first output from a transition state model, wherein the first output indicates a first state of a dynamic system; obtain a second output from the transition state model, wherein the second output indicates a state transition from the first state to a second state; retrieve a transition confidence metric associated with the state transition from a transition confidence matrix for the dynamic system based on the first state and the second state; and when the transition confidence metric indicates that the state transition is valid, provide the second state as output.

[0075] Example 2 includes the system of Example 1, wherein the instructions are further executable by the one or more processors to wait for additional output from the transition state model without providing the output when the transition confidence metric indicates that the state transition is invalid.

[0076] Example 3 includes a system according to Example 1 or Example 2, the system further including a control system, wherein the control system generates one or more control signals for an instance of the dynamic system based on a model of a second state used in response to an output received by the control system, and wherein one or more controllers associated with the instance of the dynamic system implement the one or more control signals.

[0077] Example 4 includes the system according to any one of Examples 1 to 3, further comprising a notification system, wherein the notification system generates update information associated with the instance of the dynamic system based on receipt of the output and provides the update information to one or more output devices.

[0078] According to Example 5, a method includes: obtaining a first output from a transition state model of a dynamic system at a first device, wherein the first output indicates a first state of the dynamic system; obtaining a second output from the transition state model at the first device, wherein the second output indicates a state transition from the first state to a second state; retrieving a transition confidence metric associated with the state transition from the first state to the second state from a transition confidence matrix at the first device; and providing the second state as an output via the first device in response to the transition confidence metric indicating that the state transition is valid.

[0079] Example 6 includes the method according to Example 5, wherein the output is provided to a notification system, the notification system is configured to generate update information associated with the instance of the dynamic system based on the output, and wherein the notification system is configured to provide the update information to one or more output devices.

[0080] Example 7 includes a method according to Example 5 or Example 6, wherein the output is provided to a notification system, the notification system is configured to generate update information associated with the instance of the dynamic system based on the output, and wherein the notification system is configured to provide the update information to one or more output devices.

[0081] Example 8 includes a method according to any one of Examples 5 to 7, wherein the first device or the second device is configured to generate a transition confidence matrix, and wherein generating the transition confidence matrix includes: generating a first data set of transitions of the dynamic system based on historical data, the historical data including, for each state transition detected by a transition state model of the historical data: an identifier of a case of the dynamic system undergoing the state transition, a time of the state transition, and a final state associated with the state transition; generating a second data set of data, the data of the second data set including a single entry for each case of each different type of final state identified; processing the second data set to identify the first data set that meets one or more criteria case; processing a first data set to identify, for each state transition of the first case and based on the time of the state transition, an initial state associated with a state transition preceding a final state; for each type of state, determining a first sum of occurrences of the first case in the first data set, where the state is identified as an initial state; for each initial state and final state pairing, determining a second sum of occurrences of transitions from the initial state to the final state in the first data set for the first case; and generating entries of a transition confidence matrix, wherein an entry associated with a particular initial state and a particular final state is based on the second sum of the particular initial state and the particular final state divided by the first sum associated with the particular initial state.

[0082] Example 9 includes the method of Example 8, further comprising setting each entry of the transition confidence matrix that is less than a confidence threshold to zero.

[0083] Example 10 includes the method of Example 8 or Example 9, further comprising setting each entry of the transition confidence matrix that is greater than a confidence threshold to 1.

[0084] Example 11 includes the method of any of Examples 8 to 10, further comprising changing one or more entries to zero for entries associated with transitions known not to occur.

[0085] Example 12 includes the method of any of Examples 8 to 11, further comprising comparing the transition confidence matrix to one or more expected transitions of the dynamic system.

[0086] Example 13 includes the method of Example 12, further comprising changing one or more entries in the transition confidence matrix based on comparing the transition confidence matrix to one or more expected transitions of the dynamic system.

[0087] Example 14 includes the method of example 12 or example 13, further comprising causing the transition state model to be updated based on comparing the transition confidence matrix to one or more expected transitions of the dynamic system.

[0088] Example 15 includes the method of any of Examples 8 to 14, wherein the one or more criteria include the presence of a start state and an end state.

[0089] Example 16 includes the method of any of Examples 5 to 15, wherein the dynamic system includes flight of an aircraft, and wherein the state transition from the first state to the second state includes a transition from a first flight phase to a second flight phase for the particular aircraft.

[0090] Example 17 includes the method of Example 16, wherein the transition state model is configured to generate the first output and the second output based on Automatic Dependent Surveillance-Broadcast (ADS-B) data.

[0091] According to Example 18, an apparatus includes: a memory configured to store instructions; and a processor configured to execute the instructions to perform the method according to any one of Examples 5 to 17.

[0092] According to Example 19, a non-transitory computer readable medium stores instructions that, when executed by a processor, cause the processor to perform the method of any one of Examples 5 to 17.

[0093] According to Example 20, an apparatus includes means for performing the method according to any one of Examples 5 to 17.

[0094] According to Example 21, a non-transitory computer-readable medium includes instructions that can be executed by one or more processors to: obtain a first output from a transition state model of a dynamic system, wherein the first output indicates a first state of the dynamic system; obtain a second output from the transition state model, wherein the second output indicates a state transition from the first state to a second state; retrieve a transition confidence metric associated with the state transition from the first state to the second state from a transition confidence matrix; and in response to the transition confidence metric indicating that the state transition is valid, provide a second state as an output.

[0095] Example 22 includes a non-transitory computer-readable medium according to Example 21, wherein a transition confidence matrix is ​​generated by an apparatus configured to: generate a first data set of transitions of a dynamic system based on historical data, the historical data including, for each state transition detected by a transition state model of the historical data: an identifier of a case of the dynamic system undergoing a state transition, a time of the state transition, and a final state associated with the state transition; generate a second data set, the second data set including a single entry for each case of each different type of final state identified; process the second data set to identify a first case that satisfies one or more criteria; process the first data set to identify, for each state transition of the first case and based on the time of the state transition, an initial state associated with a state transition preceding the final state; for each type of state, determine a first sum of occurrences of a first case of an instance in the first data set, where the state is identified as an initial state; for each initial state and final state pairing, determine a second sum of occurrences of transitions from the initial state to the final state in the first data set for the first case; and generate entries of the transition confidence matrix, wherein the entry associated with a particular initial state and a particular final state is based on a second sum of the particular initial state and the particular final state divided by a first sum associated with the particular initial state.

[0096] Example 23 includes the non-transitory computer readable medium according to Example 22, wherein the apparatus includes one or more processors.

[0097] The illustrations of the examples described herein are intended to provide a general understanding of the structures of different implementations. These illustrations are not intended to serve as a complete description of all elements and features of the devices and systems utilizing the structures or methods described herein. After reviewing this disclosure, many other embodiments may be apparent to those skilled in the art. Other implementations may be utilized and derived from this disclosure, so that structural and logical replacements and changes may be made without departing from the scope of this disclosure. For example, method operations may be performed in an order different from the order shown in the figure or one or more method operations may be omitted. Therefore, this disclosure and the accompanying drawings are considered to be illustrative and not restrictive.

[0098] In addition, although specific examples have been shown and described herein, it should be understood that any subsequent arrangement designed to achieve the same or similar results may replace the specific implementation shown. This disclosure is intended to cover any and all subsequent adaptations or variations of different implementations. Combinations of the above implementations and other implementations not specifically described herein will be apparent to those skilled in the art upon reviewing this description.

[0099] The abstract of the present disclosure is submitted with the understanding that it is not used to interpret or limit the scope or meaning of the claims. In addition, in the aforementioned specific embodiments, different features may be combined together or described in a single implementation for the purpose of streamlining the present disclosure. The above examples illustrate but do not limit the present disclosure. It should also be understood that many modifications and variations are possible according to the principles of the present disclosure. As reflected in the following claims, the subject matter claimed for protection may involve features that are less than all the features of any disclosed example. Therefore, the scope of the present disclosure is defined by the appended claims and their equivalents.

Claims

1. A system for providing confidence information associated with a detected state transition of a dynamic system, comprising: one or more processors; as well as a memory comprising instructions executable by the one or more processors to: obtaining a first input from a transition state model, wherein the first input indicates a first state of a dynamic system; obtaining a second input from the transition state model, wherein the second input indicates a state transition from the first state to a second state; retrieving a transition confidence metric associated with the state transition from a transition confidence matrix for the dynamic system based on the first state and the second state; and When the transition confidence metric indicates that the state transition is valid, the second state is provided as an output.

2. The system according to claim 1, wherein: The instructions are further executable by the one or more processors to wait for additional input from the transition state model without providing the output when the transition confidence metric indicates that the state transition is invalid.

3. The system according to claim 1, further comprising a control system, wherein: The control system generates one or more control signals for the instance of the dynamic system based on the model of the second state used in response to the control system receiving the output, and wherein one or more controllers associated with the instance of the dynamic system implement the one or more control signals.

4. The system according to claim 1, further comprising a notification system, wherein: The notification system generates update information associated with the instance of the dynamic system based on receipt of the output and provides the update information to one or more output devices.

5. A method for providing confidence information associated with a detected state transition of a dynamic system, comprising: obtaining, at a first device, a first input from a transition state model of a dynamic system, wherein the first input indicates a first state of the dynamic system; obtaining, at the first device, a second input from the transition state model, wherein the second input indicates a state transition from the first state to a second state; retrieving, at the first device, from a transition confidence matrix, a transition confidence metric associated with the state transition from the first state to the second state; as well as In response to the transition confidence metric indicating that the state transition is valid, providing the second state as an output via the first device.

6. The method according to claim 5, wherein: The output is provided to a control system, the control system configured to generate a control signal based on a model of the second state, and wherein the control system is configured to send the control signal to one or more controllers to control the instance of the dynamic system.

7. The method according to claim 5, wherein: The output is provided to a notification system, the notification system being configured to generate update information associated with an instance of the dynamic system based on the output, and wherein the notification system is configured to provide the update information to one or more output devices.

8. The method according to claim 5, wherein: The first device or the second device is configured to generate the transition confidence matrix, and wherein generating the transition confidence matrix comprises: generating a first data set of transitions of the dynamic system based on historical data, the historical data comprising, for each state transition detected by the transition state model for the historical data: an identifier of a case of the dynamic system undergoing a state transition, a time of the state transition, and a final state associated with the state transition; generating a second data set of data, the second data set comprising a single entry for each case of each different type of identified end state; processing the second data set to identify first cases that meet one or more criteria; processing the first data set to identify, for each state transition of the first case and based on the timing of the state transition, an initial state associated with the state transition that precedes the final state; determining, for each type of state, a first sum of occurrences of the first case in the first data set, wherein the state is identified as the initial state; for each initial state and final state pair, determining a second sum of occurrences of transitions from the initial state to the final state for the first case in the first data set; and Entries of the transition confidence matrix are generated, wherein an entry associated with a particular initial state and a particular final state is based on the second sum of the particular initial state and the particular final state divided by the first sum associated with the particular initial state.

9. The method according to claim 8, further comprising: Each entry of the transition confidence matrix that is less than a confidence threshold is set to zero.

10. The method according to claim 8, further comprising: Each entry of the transition confidence matrix that is greater than a confidence threshold is set to 1.

11. The method according to claim 8, further comprising: For entries associated with transitions that are known not to occur, one or more entries are changed to zero.

12. The method according to claim 8, further comprising: The transition confidence matrix is ​​compared to one or more expected transitions of the dynamic system.

13. The method according to claim 12, further comprising: One or more entries in the transition confidence matrix are changed based on comparing the transition confidence matrix to the one or more expected transitions of the dynamic system.

14. The method according to claim 12, further comprising: The transition state model is caused to update based on comparing the transition confidence matrix to the one or more expected transitions of the dynamic system.

15. The method according to claim 8, wherein: The one or more criteria include the presence of a start state and an end state.

16. The method according to claim 5, wherein: The dynamic system includes flight of an aircraft, and wherein the state transition from the first state to the second state includes a transition from a first flight phase to a second flight phase for a particular aircraft.

17. The method according to claim 16, wherein: The transition state model is configured to generate the first input and the second input based on Automatic Dependent Surveillance-Broadcast (ADS-B) data.

18. A non-transitory computer readable medium comprising instructions executable by one or more processors to: The first input is obtained from a transition state model of the dynamic system, where The first input indicates a first state of the dynamic system; obtaining a second input from the transition state model, wherein the second output indicates a state transition from the first state to a second state; retrieving a transition confidence metric associated with the state transition from the first state to the second state from a transition confidence matrix; and In response to the transition confidence metric indicating that the state transition is valid, providing the second state as an output.

19. The non-transitory computer readable medium of claim 18, wherein: The transition confidence matrix is ​​generated by an apparatus configured to: generating a first data set of transitions of the dynamic system based on historical data, the historical data comprising, for each state transition detected by the transition state model for the historical data: an identifier of a case of the dynamic system undergoing a state transition, a time of the state transition, and a final state associated with the state transition; generating a second data set comprising a single entry for each case of each different type of identified end state; processing the second data set to identify first cases that meet one or more criteria; processing the first data set to identify, for each state transition of the first case and based on the timing of the state transition, an initial state associated with the state transition that precedes the final state; determining, for each type of state, a first sum of occurrences of the first case in the first data set, wherein the state is identified as the initial state; for each initial state and final state pair, determining a second sum of occurrences of transitions from the initial state to the final state for the first case in the first data set; and Entries of the transition confidence matrix are generated, wherein an entry associated with a particular initial state and a particular final state is based on the second sum of the particular initial state and the particular final state divided by the first sum associated with the particular initial state.

20. The non-transitory computer readable medium of claim 19, wherein: The apparatus comprises the one or more processors.

Images