Achieving computer system integrity through a combination of certifiable and qualified software

Abstract

The present invention relates to achieving the integrity of a computer system through a combination of certifiable and qualified software. A method for improving the integrity of a computer system includes executing a certifiable and qualified software application. The certifiable software application consists of static program instructions that are executed sequentially to process input data to produce outputs, and the qualified software application uses a model that is iteratively constructed using a machine learning algorithm to process the input data to produce corresponding outputs. According to the certification standard, the certifiable software application is certifiable for the computer system, and according to the certification standard, the qualified software application is not certifiable for the computer system. The method also includes reviewing the output to verify the output by comparing it with the corresponding output, and thereby improving the integrity of the computer system. And the method includes generating an alarm that the output is not verified when the comparison indicates that the output differs from the corresponding output by more than a threshold.

Description

Technical Field

[0001] The present disclosure relates generally to computer systems, and more particularly, to combining certifiable and qualified software to improve computer system integrity. Background Art

[0002] In many industries, such as aerospace, a high level of integrity is required. For many of these industries, software is required to meet various certification standards or safety levels. For example, in aerospace, the DO-178 requirements promulgated by the Radio Technical Commission for Avionics (RTCA) provide a number of different aircraft-related certification levels for software / hardware systems. Certification is designed to ensure that the software is correct and only operates as designed. This usually means starting with a certifiable operating system (OS) and compiler that can accurately translate high-level code into machine language. Then the application code is tested and checked line by line. But even this does not mean the results will meet expectations.

[0003] Image processing provides a good example of the problem. While many high-integrity systems operate on one or two time-varying signals that can be well characterized, video systems are inherently more complex. Input images (video) consist of millions of pixels. One result is that it is difficult to fully characterize all possible inputs. Even if the software can be certified by other means, it is equally difficult to ensure that any image or video processing algorithm will return the "correct" results under all conditions. One way to often mitigate this uncertainty is to have trained (qualified) human operators provide oversight.

[0004] In certain high integrity situations (such as flight control systems), additional integrity is obtained by having multiple (usually three) parallel and independent but identical computers running the same software on the same inputs. In this case, the outputs are expected to match exactly, and comparison operations on the outputs are used to ensure that all computers produce the same results.

[0005] Artificial intelligence (AI / ML) based on machine learning takes a very different approach. In some examples, AI / ML software implements a multi-layer, multi-node "neural network" that is trained with labeled example data sets. The learning system adjusts the connections between nodes to "learn" a topology that provides correct results. Although these systems can be "frozen" so that the behavior is fixed rather than evolving, it is still not feasible to follow a linear thread of execution as is done for certifiable systems. As a result, these powerful systems cannot currently be certified. Current discussions about how to utilize these systems in high-integrity applications involve the idea of ​​"qualifying the system" just as people are "qualified" through training and testing / evaluation in various situations. But this approach is not yet acceptable in systems that require high integrity.

[0006] It would therefore be desirable to have a system and method that takes into account at least some of the issues discussed above as well as other possible issues. Summary of the invention

[0007] Example embodiments of the present disclosure relate to combining certifiable and qualified software to improve computer system integrity. Specifically, some example embodiments utilize non-certifiable machine learning software to review certifiable software and thereby improve computer system integrity. In some examples, outputs from certifiable software may simply be reviewed with corresponding outputs from qualified software.

[0008] In some examples, the output and the corresponding output can be combined to produce a combined output, thereby improving not only the integrity but also the robustness of the computer system. In this regard, different software can provide similar but slightly different outputs with corresponding quality factors. If the outputs are consistent to a sufficient degree, they can be weighted (by quality factors) and combined (e.g., in a Kalman filter) to provide improved performance.

[0009] Accordingly, the present disclosure includes, but is not limited to, the following example embodiments.

[0010] Some example embodiments provide a method for improving the integrity of a computer system, the method comprising: the computer system executing a certifiable software application consisting of static program instructions, which are executed sequentially to process input data to produce output, the computer system being configured to operate on the output, the certifiable software application being certifiable to the computer system according to a certification standard; executing a qualified software application during the execution of the certifiable software application, the qualified software application processing the input data using a model iteratively constructed by using a machine learning algorithm to produce a corresponding output, the qualified software application being uncertifiable to the computer system according to the certification standard; reviewing the output from the certifiable software application by comparing it with a corresponding output from the qualified software application to verify the output from the certifiable software application and thereby improve the integrity of the computer system; and when the comparison indicates that the output differs from the corresponding output by more than a threshold, generating an alarm indicating that the output from the certifiable software application is not verified.

[0011] In some example embodiments of the method of any preceding example embodiment or any combination of any preceding example embodiments, the output from the certifiable software application is verified when the comparison indicates that the output does not differ from the corresponding output from the qualified software application by more than a threshold, and the method further includes the computer system operating on the output that the computer system is configured to operate on and excluding the corresponding output from the qualified software application.

[0012] In some example embodiments of the method of any preceding example embodiment or any combination of any preceding example embodiments, the output from the certifiable software application is verified when the comparison indicates that the output does not differ from the corresponding output from the qualified software application by more than a threshold, and the method further includes the computer system generating a combined output from the output and the corresponding output; and operating on the combined output.

[0013] In some example embodiments of the method of any preceding example embodiment or any combination of any preceding example embodiments, executing the certifiable software application and executing the qualified software application further includes generating corresponding quality factors for the output and the corresponding output, and wherein producing the combined output includes Kalman filtering the output and the corresponding output weighted by the corresponding quality factors.

[0014] In some example embodiments of the method of any preceding example embodiment or any combination of any preceding example embodiments, while executing the certifiable software application and the qualified software application, a third software application is executed, the third software application being a second certifiable or qualified software application, the third software application being executed to process input data to produce a second corresponding output, wherein reviewing the output includes reviewing the output from the certifiable software application by comparing it with the corresponding output and the second corresponding output, and wherein generating an alarm includes generating an alarm that the output from the certifiable software is not verified when the comparison indicates that the output differs from both the corresponding output and the second corresponding output by more than a threshold.

[0015] In some example embodiments of the method of any preceding example embodiment or any combination of any preceding example embodiments, the computer system is an avionics system of an aircraft moving toward a hub defining an aircraft target, the certifiable software application is a certifiable object detection application, and the qualified software application is a qualified object detection application, wherein executing the certifiable software application includes executing the certifiable object detection application to process an image of the hub to produce an output in which the target is detected in the image, wherein executing the qualified software application includes executing the qualified object detection application to process an image of the hub to produce a corresponding output in which the target is detected in the image, and wherein reviewing the output includes reviewing the output by comparing with the corresponding output to verify the output from the certifiable object detection application.

[0016] In some example embodiments of the method of any preceding example embodiment or any combination of any preceding example embodiments, the output from the certifiable object detection application is validated when the comparison indicates that the output differs from the corresponding output by no more than a threshold that decreases with distance between the aircraft and the hub.

[0017] Some example embodiments provide a computer system comprising a memory configured to store a computer-readable program code; and a processing circuit configured to access the memory and execute the computer-readable program code to cause the computer system to perform at least the method of any previous example embodiment or any combination of any previous example embodiment.

[0018] Some example embodiments provide a computer-readable storage medium that is non-transitory and has a computer-readable program code stored therein, which, in response to execution by a processing circuit, causes a computer system to at least perform the method of any of the foregoing example embodiments, or any combination of any of the foregoing example embodiments.

[0019] These and other features, aspects and advantages of the present disclosure will become apparent by reading the following detailed description and the accompanying drawings briefly described below. The present disclosure includes any combination of two, three, four or more features or elements set forth in the present disclosure, regardless of whether these features or elements are explicitly combined or otherwise narrated in the specific example embodiments described herein. It is intended to understand the present disclosure as a whole, so that in any aspect and example embodiment of the present disclosure, any separable feature or element of the present disclosure should be considered combinable, unless the context of the present disclosure clearly indicates otherwise.

[0020] Therefore, it will be understood that the present disclosure is provided only for the purpose of summarizing some example embodiments in order to provide a basic understanding of some aspects of the present disclosure. Therefore, it will be appreciated that the above-mentioned example embodiments are merely examples and should not be interpreted as narrowing the scope or spirit of the present disclosure in any way. Other example embodiments, aspects and advantages will become apparent from the following detailed description in conjunction with the accompanying drawings, which illustrate, by way of example, the principles of some of the described example embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Having thus described example embodiments of the present disclosure in general terms, reference will now be made to the accompanying drawings, which are not necessarily drawn to scale, and in which:

[0022] Figure 1 is a functional block diagram of a computer system with improved integrity according to an example implementation of the present disclosure;

[0023] Figure 2 Automatic aerial refueling is shown according to various example embodiments.

[0024] Figure 3 is a flowchart illustrating various steps in a method of improving the integrity of a computer system according to various example embodiments; and

[0025] Figure 4 A computer system according to some example embodiments is shown. DETAILED DESCRIPTION

[0026] Some embodiments of the present disclosure will now be described more fully below with reference to the accompanying drawings, in which some but not all embodiments of the present disclosure are shown. In fact, various embodiments of the present disclosure may be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein; on the contrary, these example embodiments are provided so that the present disclosure will be thorough and complete, and will fully convey the scope of the present disclosure to those skilled in the art. For example, unless otherwise stated, references to first, second, etc. should not be interpreted as implying a particular order. Similarly, certain things may be described as being above other things (unless otherwise stated), but may be below them, or vice versa; similarly, things described as being on the left side of other things may be on the right side, or vice versa. Throughout the text, the same reference numerals represent the same elements.

[0027] Example embodiments of the present disclosure are generally related to computer systems, and in particular, to combining certifiable and qualified software to improve computer system integrity. The computer system can be any of a plurality of different computer systems in which software can be certified according to certification standards. Specifically, the computer system can be an avionics system carried by a manned or unmanned aircraft, a satellite, a spacecraft, etc. Examples of suitable avionics systems include avionics systems for flight control systems, air positioning systems, automatic take-off or landing systems, rendezvous and docking systems, or any other high-integrity systems with strict safety requirements. In other examples, the computer system can be a computer system used outside of aviation. Examples include automated surgical procedures, computer-controlled medical devices, automated ground or sea vehicles and functions. Other examples include factory automation or inspection and maintenance services, in which robots or computer-controlled systems interact with humans or interact near humans.

[0028] Certifiable software is a software application (certifiable software application) typically consisting of static program instructions that are executed sequentially to process input data to produce output upon which the computer system is configured to operate. Typical examples are vehicle management systems or flight control computers that convert steering and throttle commands into movement of aircraft surfaces. Other examples more generally include any system that protects the flight envelope or provides safety-critical information to personnel where the presentation of erroneous information could cause the operator to select an unsafe alternative.

[0029] A certifiable software application is certifiable for a computer system according to a certification standard; and when certified, a certifiable software application may be referred to as a certified software application. Examples of suitable certifiable software include embedded software in avionics systems, at least some of which may be certifiable according to DO-178. This particularly includes avionics software, which is typically embedded software used in avionics and has legally required safety and reliability. Other examples of suitable certifiable software include software used on manned or unmanned aircraft, satellites, spacecraft, etc., which does not have legally required safety and reliability.

[0030] Qualifiable software is generally a software application (qualified software application) that is not certifiable for a computer system according to existing certification standards. As described primarily herein, qualified software uses machine learning to generate outputs corresponding to the outputs of a certifiable software application. Specifically, a qualified software application uses a model iteratively constructed by a machine learning algorithm to process input data to generate corresponding outputs. An example of a suitable machine learning algorithm is an artificial neural network (ANN). Other examples include regression algorithms, decision trees, support vector machines, Bayesian networks, and the like. Relative to certifiable software applications, qualified software applications are not certifiable for computer systems according to certification standards. Qualified software applications are qualified for computer systems, just as people are "qualified" by training and testing / evaluation in various situations. And when qualified, qualified software applications can be referred to as qualified software applications.

[0031] Figure 1 is a functional block diagram of a computer system 100 according to an example implementation of the present disclosure. The computer system may include any of a plurality of different functional blocks for performing one or more functions or operations. As shown, in some examples, the computer system includes a certifiable execution block 102, a qualified execution block 104, a cross-check block 106, an alarm block 108, and an operation block 110. The functional blocks including the certifiable execution block, the qualified execution block, the cross-check block, the alarm block, and the operation block may be co-located or directly coupled to each other, or in some examples, the various functional blocks may communicate with each other on one or more computer networks. In addition, although shown as part of a computer system, it should be understood that any one or more of the above may be outside the computer system, without regard to any other functional blocks. It should also be understood that the computer system may include other than Figure 1 One or more additional or alternative functional blocks than those shown in .

[0032] In some example embodiments, the certifiable executable block 102 is configured to execute a certifiable software application 112. As described above, the certifiable software application consists of static program instructions that are executed sequentially to process input data 114, such as sensor data, to produce output 116, and the computer system 100 is configured to operate on the output 116. According to the certification standard, the certifiable software application is certifiable for the computer system.

[0033] Qualified execution block 104 is configured to execute qualified software application 118 during execution of certifiable software application 112. The qualified software application processes input data 114 using a model iteratively built by a machine learning algorithm to produce corresponding output 120. According to the certification standard, the qualified software application is not certifiable for the computer system.

[0034] The review block 106 is configured to review the output 116 from the certifiable software application 112 by comparing it with the corresponding output 120 from the qualified software application 118 to verify the output from the certifiable software application and thereby improve the security of the computer system 100. The alarm block 108 is configured to generate an alarm 122 that the output from the certifiable software application is not verified when the comparison indicates that the output differs from the corresponding output by more than a threshold. The threshold may depend on the type of computer system and / or certifiable software application and may define an acceptable variation or tolerance in the output from the certifiable software application.

[0035] When the comparison indicates that the output does not differ from the corresponding output 120 from the qualified software application 118 by more than a threshold, the output 116 from the certifiable software application 112 is verified. In these examples, the computer system 100 can operate on the output (the computer system is configured to operate on the output). To this end, the computer system also includes an operation block 110 configured to operate on the output. The operation generally depends on the type of computer system, such as the type of avionics system implementing the exemplary embodiments of the present disclosure. Examples include generating trajectories based on input sensor data and then passing such trajectories to a vehicle management system.

[0036] In some examples, the qualified software application 118 is only used to verify the certifiable software application 112. In these examples, the operation block 110 is configured to operate on the output 116 without including the corresponding output 120 from the qualified software application. In other examples, the qualified software application can be further used to improve the accuracy of the certifiable software application. Here, the computer system 100 also includes a combination block 124, which is configured to generate a combined output from the output and the corresponding output. Then, the operation block is configured to operate on the combined output.

[0037] In some other examples, the certifiable execution block 102 and the qualified execution block 104 are further configured to generate respective figures of merit for the output 116 and the corresponding output 120. In some of these examples, the combination block 124 is configured to Kalman filter the output and the corresponding output weighted by the respective figures of merit to generate a combined output.

[0038] Although described as including the certifiable software application 112 and the qualified software application 118, it should be understood that the computer system 100 can be configured to execute more than two software applications for use in the example embodiments of the present disclosure. That is, the computer system can utilize three or more software applications. Specifically, in some examples, the computer system 100 also includes a third execution block 126, which is configured to execute the third software application 128 when executing the certifiable software application 112 and the qualified software application 118.

[0039] In examples that include a third software application 128 (or more), the third software application is a second certifiable or qualified software application, or some other software application. The third software application is executed to process the input data 114 to produce a second corresponding output 130. The review block 106 is then configured to review the output 116 from the certifiable software application 112 by comparing it with the corresponding output 120 and the second corresponding output 130. The alarm block 108 is configured to generate an alarm 122 that the output from the certifiable software is not verified when the comparison indicates that the output differs from both the corresponding output and the second corresponding output by more than a threshold.

[0040] As indicated above, computer system 100 can be any of a number of different computer systems. In some examples, the computer system is an avionics system onboard a manned or unmanned aircraft, satellite, spacecraft, or the like. In this context, automated air-to-air refueling (A3R) and automated spacecraft rendezvous and docking involve bringing two aerospace vehicles together with sufficient precision to complete a connection maneuver. It is often important that the automated system identifies the correct target for the connection. To illustrate, an automated refueling aircraft attempting to perform an A3R by inserting its probe into a tanker drogue must ensure that the circular feature identified is actually the drogue, and not exhaust from an engine. Similarly, an automated refueling boom must ensure that it is attempting to insert into the refueling tube and not into a cockpit window.

[0041] In many high integrity cases, integrity can be achieved by using multiple data sources (e.g., multiple sensors providing different viewing geometries or even different sensing technologies such as passive imaging and 3D LIDAR) and by using multiple algorithms for each data source. In the case of A3R, two passive imaging cameras can be used, each imaging the same area, but mounted in different locations to provide multiple source data (if some objects occlude one camera, the area is still visible in the other camera). For each camera, a certifiable software application and a qualified software application can be executed on the input image data, and their respective outputs compared.

[0042] In some examples, the tolerance (threshold) decreases with the distance between the two aircraft, but as long as both software applications are within the tolerance, operation can continue. Depending on the requirements, the qualified software application can be used only as a check on the certifiable software application. In these examples, as long as the two software applications are consistent, the output from the certifiable software application is used. If they are inconsistent, the computer system will declare an error, generate an alarm, and then the operation will terminate. In some examples where the software applications are consistent, their respective outputs can be input into the Kalman filter to improve the overall accuracy.

[0043] Figure 2 Automated aerial refueling 200 is shown according to some examples. In these examples, computer system 100 is an avionics system 202 of an aircraft 204, which moves toward a hub 206 defining a target 208 of the aircraft. As shown, the hub is a fuel replenishment or refueling aircraft (or simply "tanker"), and the target of the aircraft is a drogue extending from an extension hose to the rear of the tanker. The aircraft carries a refueling probe 210 and approaches the hub, for example, from the tail and below, in an effort to engage the refueling probe with the drogue and subsequently provide aerial refueling to the aircraft. Here, certifiable software application 112 and qualified software application 118 can be a certifiable object detection application and a qualified object detection application, respectively.

[0044] The certifiable execution block 102 can be configured to execute a certifiable object detection application to process an image of the hub 206 to produce an output in which a target 208 is detected in the image. Similarly, the qualified execution block 104 can be configured to execute a qualified object detection application to process an image of the hub to produce a corresponding output in which a target is detected in the image. The review block 106 can be configured to review the output by comparing it with the corresponding output to verify the output from the certifiable object detection application. And in some examples, the output is verified when the comparison indicates that the output does not differ from the corresponding output by more than a threshold value that decreases with the distance between the aircraft and the hub. More information about systems for automatic aerial refueling that can benefit from example embodiments of the present disclosure is provided in U.S. Patent No. 9,933,521, "AerialPositioning Systems and Methods," the contents of which are incorporated herein by reference in their entirety.

[0045] Figure 3 300 is a flowchart illustrating various steps in a method 300 for improving the integrity of a computer system according to an example embodiment of the present disclosure. As shown in step 302, the method includes executing a certifiable software application consisting of static program instructions that are sequentially executed to process input data to produce output, and the computer system is configured to operate on the output. In this regard, the certifiable software application is certifiable for the computer system according to the certification standard.

[0046] Method 300 includes executing a qualified software application during execution of a certifiable software application, as shown in step 304. The qualified software application processes input data using a model iteratively built by a machine learning algorithm to produce corresponding outputs. According to the certification standard, the qualified software application is not certifiable for the computer system.

[0047] The method 300 includes, as shown in step 306, reviewing the output from the certifiable software application by comparing it with the corresponding output from the qualified software application to verify the output from the certifiable software application and thereby improve the integrity of the computer system. As shown in step 308, the method includes: when the comparison indicates that the output differs from the corresponding output by more than a threshold, generating an alarm that the output from the certifiable software application is not verified. When the comparison indicates that the output does not differ from the corresponding output from the qualified software application by more than the threshold, the output from the certifiable software application is verified. Then, in some examples, as shown in step 310, the method also includes operating on the output or the combined output.

[0048] According to example embodiments of the present disclosure, the computer system 100 and its functional blocks including the authenticatable execution block 102, the qualified execution block 104, the review block 106, the alarm block 108, the operation block 110, the combination block 124 and the third execution block 126 can be implemented in various ways. The device for implementing the computer system and its functional blocks may include separate hardware or hardware under the guidance of one or more computer programs from a computer-readable storage medium. In some examples, the computer system may be embodied by one or more devices configured to act as or otherwise implement the functional blocks shown and described herein. In examples involving more than one device, the various devices may be connected to each other or otherwise communicate in a variety of different ways, such as directly or indirectly via a wired or wireless network, etc.

[0049] Figure 4 An apparatus 400 according to some example embodiments of the present disclosure is shown. In general, an apparatus of an example embodiment of the present disclosure may include, comprise, or be embodied in one or more fixed or portable electronic devices. Examples of suitable electronic devices include smart phones, tablet computers, laptop computers, desktop computers, workstation computers, server computers, etc. For certifiable avionics systems, suitable electronic devices typically consist of a certifiable electronic chassis with a backplane and one or more certifiable processor cards. The apparatus may include one or more of each of a plurality of components, for example, a processing circuit 402 (e.g., a processor unit) connected to a memory 404 (e.g., a storage device).

[0050] Processing circuitry 402 may be comprised of one or more processors, alone or in combination with one or more memories. Processing circuitry is typically any computer hardware capable of processing information such as data, computer programs, and / or other suitable electronic information. Processing circuitry is comprised of a collection of electronic circuits, some of which may be packaged as an integrated circuit or multiple interconnected integrated circuits (integrated circuits are sometimes more generally referred to as "chips"). Processing circuitry may be configured to execute a computer program, which may be stored on the processing circuitry or otherwise stored in memory 404 (of the same or another device).

[0051] Depending on the specific implementation, the processing circuit 402 may be a plurality of processors, a multi-core processor, or some other type of processor. The processing circuit may include a graphics processing unit (GPU), a central processing unit (CPU), or a combination of a GPU and a CPU. In addition, a plurality of heterogeneous processor systems may be used to implement the processing circuit, wherein the main processor is present on a single chip together with one or more auxiliary processors. As another illustrative example, the processing circuit may be a symmetric multiprocessor system comprising a plurality of processors of the same type. In yet another example, the processing circuit may be embodied as or otherwise include one or more ASICs, FPGAs, etc. Therefore, although the processing circuit is capable of executing a computer program to perform one or more functions, the processing circuits of various examples are capable of performing one or more functions without the aid of a computer program. In either case, according to an example implementation of the present disclosure, the processing circuit may be appropriately programmed to perform a function or operation.

[0052] Memory 404 is generally any computer hardware capable of temporarily and / or permanently storing information, such as data, computer programs (e.g., computer readable program code 406), and / or other suitable information. Memory may include volatile and / or non-volatile memory, and may be fixed or removable. Examples of suitable memory include random access memory (RAM), read-only memory (ROM), hard drive, flash memory, thumb drive, removable computer disk, optical disk, tape, or some combination thereof. The optical disk may include compact disc-read only memory (CD-ROM), compact disc-read / write (CD-R / W), DVD, etc. In various cases, memory may be referred to as a computer-readable storage medium. The computer-readable storage medium is a non-temporary device capable of storing information, and is distinguished from a computer-readable transmission medium such as an electronic transient signal capable of carrying information from one location to another. Computer-readable media as described herein may generally refer to a computer-readable storage medium or a computer-readable transmission medium.

[0053] In addition to the memory 404, the processing circuit 402 may also be connected to one or more interfaces for displaying, transmitting and / or receiving information. The interface may include a communication interface 408 (e.g., a communication unit). The communication interface may be configured to transmit and / or receive information, such as to and / or from other (multiple) devices, (multiple) networks, etc. The communication interface may be configured to transmit and / or receive information via a physical (wired) and / or wireless communication link. Examples of suitable communication interfaces include a network interface controller (NIC), a wireless NIC (WNIC), etc.

[0054] In some examples, the interface may include one or more user interfaces, such as a display 410 and / or one or more user input interfaces 412 (e.g., an input / output unit). The display may be configured to present or otherwise display information to a user, suitable examples of which include a liquid crystal display (LCD), a light emitting diode display (LED), a plasma display panel (PDP), etc. The user input interface may be wired or wireless, and may be configured to receive information from a user into the device, such as for processing, storage, and / or display. Suitable examples of user input interfaces include a microphone, an image or video capture device, a keyboard or keypad, a joystick, a touch-sensitive surface (separate from or integrated into a touch screen), a biometric sensor, etc. The user interface may also include one or more interfaces for communicating with peripheral devices such as a printer, a scanner, etc.

[0055] As indicated above, program code instructions can be stored in a memory and executed by a processing circuit programmed thereby to implement the functions of the systems, subsystems, tools and their corresponding elements described herein. It will be understood that any suitable program code instructions can be loaded from a computer-readable storage medium onto a computer or other programmable device to produce a specific machine so that the specific machine becomes a device for implementing the functions specified herein. These program code instructions can also be stored in a computer-readable storage medium, which can guide a computer, a processing circuit or other programmable device to function in a particular manner, thereby generating a specific machine or a specific product. Instructions stored in a computer-readable storage medium can produce products, wherein the products become devices for implementing the functions described herein. Program code instructions can be retrieved from a computer-readable storage medium and loaded into a computer, a processing circuit or other programmable device to configure a computer, a processing circuit or other programmable device to perform operations to be performed on a computer, a processing circuit or by it.

[0056] The retrieval, loading, and execution of program code instructions may be performed sequentially so that one instruction is retrieved, loaded, and executed at a time. In some example embodiments, the retrieval, loading, and / or execution may be performed in parallel so that multiple instructions are retrieved, loaded, and / or executed together. The execution of program code instructions may produce a computer-implemented process such that the instructions executed by a computer, processing circuit, or other programmable device provide operations for implementing the functions described herein.

[0057] The combination of operations for performing the specified functions is supported by executing instructions through processing circuitry or storing instructions in a computer-readable storage medium. In this manner, the device 400 may include a processing circuit 402 and a computer-readable storage medium or memory 404 coupled to the processing circuit, wherein the processing circuit is configured to execute a computer-readable program code 406 stored in the memory. It should also be understood that one or more functions and combinations of functions may be implemented by a dedicated hardware-based device and / or processing circuitry that performs a particular function, or may be implemented by a combination of dedicated hardware and program code instructions.

[0058] Furthermore, the present disclosure includes embodiments according to the following items:

[0059] Item 1. A computer system (400), comprising:

[0060] a memory (404) configured to store computer readable program code (406); and

[0061] The processing circuit (402) is configured to access the memory and execute the computer readable program code to cause the computer system to at least:

[0062] executing (302) a certifiable software application consisting of static program instructions that are sequentially executed to process input data to produce output on which the computer system is configured to operate, the certifiable software application being certifiable to the computer system according to a certification standard;

[0063] executing (304) during execution of the certifiable software application a qualifying software application that processes input data using a model iteratively constructed by a machine learning algorithm to produce corresponding outputs, the qualifying software application being non-certifiable for the computer system according to the certification criteria;

[0064] Reviewing (306) the output from the certifiable software application by comparing it with the corresponding output from the qualified software application to verify the output from the certifiable software application and thereby improve the integrity of the computer system; and

[0065] When the comparison indicates that the output differs from the corresponding output by more than a threshold, an alert is generated (308) that the output from the certifiable software application is not authenticated.

[0066] The computer system (400) of item 2 or item 1, wherein when the comparison indicates that the output does not differ from the corresponding output from the qualified software application by more than a threshold, the output from the certifiable software application is verified, and the memory (404) stores additional computer readable program code (406), and the processing circuit (402) is configured to execute the additional code so that the computer system further at least:

[0067] Operations are performed on outputs that the computer system is configured to perform operations on (310), and corresponding outputs from qualifying software applications are excluded.

[0068] Item 3. The computer system (400) of any preceding item, wherein when the comparison indicates that the output does not differ from the corresponding output from the qualified software application by more than a threshold, the output from the certifiable software application is verified, and the memory (404) stores additional computer readable program code (406), and the processing circuit (402) is configured to execute the additional code to cause the computer system to further at least:

[0069] generating a combined output from the output and the corresponding output; and

[0070] Operate on combined output.

[0071] The computer system (400) of item 4 and item 3, wherein causing the computer system to execute (302) a certifiable software application and to execute (304) a qualified software application further comprises causing the computer system to generate a corresponding quality factor for the output and the corresponding output, and

[0072] Therein, causing the computer system to generate a combined output includes performing Kalman filtering on the output weighted by the corresponding quality factor and the corresponding output.

[0073] Item 5. The computer system (400) of any preceding item, wherein the memory (404) stores additional computer-readable program code (406), and the processing circuit (402) is configured to execute the additional code to cause the computer system to further at least:

[0074] executing a third software application, the third software application being the second certifiable or qualified software application, when the certifiable software application and the qualified software application are executed, the third software application being executed to process the input data to produce a second corresponding output,

[0075] wherein causing the computer system to review (306) the output includes causing the computer system to review the output from the authenticatable software application by comparing the corresponding output and a second corresponding output, and

[0076] Wherein causing the computer system to generate (308) an alert includes causing the computer system to generate an alert that the output from the certifiable software is not verified when the comparison indicates that the output differs from both the corresponding output and the second corresponding output by more than a threshold.

[0077] Item 6. The computer system (400) of any preceding item, wherein the computer system is an avionics system of an aircraft moving toward a hub defining an aircraft target, the certifiable software application is a certifiable object detection application, and the qualified software application is a qualified object detection application,

[0078] wherein causing the computer system to execute (302) the authenticatable software application comprises causing the computer system to execute the authenticatable object detection application to process the image of the hub to produce an output in which an object is detected in the image,

[0079] wherein causing the computer system to execute (304) a qualified software application comprises causing the computer system to execute a qualified object detection application to process the image of the hub to produce a corresponding output in which an object is detected in the image, and

[0080] Wherein causing the computer system to review the output (306) includes causing the computer system to review the output by comparing it with a corresponding output to verify the output from the authenticatable object detection application.

[0081] Item 7. The computer system (400) of item 6, wherein the output from the authenticatable object detection application is verified when the comparison indicates that the output differs from the corresponding output by no more than a threshold that decreases with the distance between the aircraft and the hub.

[0082] Item 8. A method (300) of improving the integrity of a computer system (400), the method comprising: the computer system:

[0083] executing (302) a certifiable software application consisting of static program instructions that are sequentially executed to process input data to produce output on which the computer system is configured to operate, the certifiable software application being certifiable to the computer system according to a certification standard;

[0084] executing (304) during execution of the certifiable software application a qualifying software application that processes input data using a model iteratively constructed by a machine learning algorithm to produce corresponding outputs, the qualifying software application being non-certifiable for the computer system according to the certification criteria;

[0085] Reviewing (306) the output from the certifiable software application by comparing it with the corresponding output from the qualified software application to verify the output from the certifiable software application and thereby improve the integrity of the computer system; and

[0086] When the comparison indicates that the output differs from the corresponding output by more than a threshold, an alert is generated (308) that the output of the certifiable software application is not verified.

[0087] Item 9. The method (300) of Item 8, wherein the output from the certifiable software application is verified when the comparison indicates that the output does not differ from the corresponding output from the qualified software application by more than a threshold, and the method further comprises the computer system:

[0088] Operations are performed (310) on the outputs that the computer system is configured to operate on, and corresponding outputs from qualifying software applications are excluded.

[0089] The method (300) of any one of clauses 10, 8 to 9, wherein the output from the certifiable software application is verified when the comparison indicates that the output does not differ from the corresponding output from the qualified software application by more than a threshold, and the method further comprises the computer system:

[0090] generating a combined output from the output and the corresponding output; and

[0091] Operate on combined output.

[0092] Item 11. The method (300) of item 10, wherein executing (302) a certifiable software application and executing (304) a qualified software application further comprises generating a corresponding quality factor for the output and the corresponding output, and

[0093] Wherein generating the combined output comprises performing Kalman filtering on the output weighted by the corresponding quality factor and the corresponding output.

[0094] Item 12, the method (300) of any one of Items 8 to 11, further comprising:

[0095] executing a third software application, the third software application being the second certifiable or qualified software application, when the certifiable software application and the qualified software application are executed, the third software application being executed to process the input data to produce a second corresponding output,

[0096] wherein reviewing (306) the output includes reviewing the output from the certifiable software application by comparing it with the corresponding output and a second corresponding output, and

[0097] Wherein generating (308) an alarm includes causing an alarm to be generated that the output from the certifiable software is not authenticated when the comparison indicates that the output differs from both the corresponding output and the second corresponding output by more than a threshold.

[0098] The method (300) of any one of clauses 13, 8 to 12, wherein the computer system is an avionics system of an aircraft moving toward a hub defining an aircraft target, the certifiable software application is a certifiable object detection application, and the qualified software application is a qualified object detection application,

[0099] wherein executing (302) the authenticatable software application includes executing the authenticatable object detection application to process an image of the hub to produce an output in which an object is detected in the image,

[0100] wherein executing (304) the qualified software application includes executing a qualified object detection application to process the image of the hub to produce a corresponding output in which an object is detected in the image, and

[0101] Wherein, verifying the output (306) includes verifying the output by comparing it with a corresponding output to verify the output from the authenticatable object detection application.

[0102] Item 14. The method (300) of item 13, wherein the output from the certifiable object detection application is verified when the comparison indicates that the output differs from the corresponding output by no more than a threshold that decreases with the distance between the aircraft and the hub.

[0103] Item 15. A non-transitory computer-readable storage medium (404) having computer-readable program code (406) stored therein, the computer-readable program code, in response to being executed by a processing circuit (402), causing a computer system (400) to at least:

[0104] executing (302) a certifiable software application consisting of static program instructions that are sequentially executed to process input data to produce output on which the computer system is configured to operate, the certifiable software application being certifiable to the computer system according to a certification standard;

[0105] executing (304) during execution of the certifiable software application a qualifying software application that processes input data using a model iteratively constructed by a machine learning algorithm to produce corresponding outputs, the qualifying software application being non-certifiable for the computer system according to the certification criteria;

[0106] Reviewing (306) the output from the certifiable software application by comparing it with the corresponding output from the qualified software application to verify the output from the certifiable software application and thereby improve the integrity of the computer system; and

[0107] When the comparison indicates that the output differs from the corresponding output by more than a threshold, an alert is generated (308) that the output of the certifiable software application is not verified.

[0108] The computer-readable storage medium (404) of item 16 or item 15, wherein the output from the certifiable software application is verified when the comparison indicates that the output does not differ from the corresponding output from the qualified software application by more than a threshold value, and the computer-readable storage medium (404) stores therein additional computer-readable program code (406) which, in response to execution by the processing circuit (402), causes the computer system (400) to further at least:

[0109] Operations are performed on outputs that the computer system is configured to perform operations on (310), and corresponding outputs from qualifying software applications are excluded.

[0110] The computer-readable storage medium (404) of any one of clauses 17, 15 to 16, wherein the output from the certifiable software application is verified when the comparison indicates that the output does not differ from the corresponding output from the qualified software application by more than a threshold, and the computer-readable storage medium (404) stores therein further computer-readable program code (406) which, in response to execution by the processing circuit (402), causes the computer system (400) to further at least:

[0111] generating a combined output from the output and the corresponding output; and

[0112] Operate on combined output.

[0113] The computer-readable storage medium (404) of item 18 and item 17, wherein causing the computer system to execute (302) a certifiable software application and to execute (304) a qualified software application further comprises causing the computer system to generate a corresponding quality factor for the output and the corresponding output, and

[0114] Therein, causing the computer system to generate a combined output includes performing Kalman filtering on the output weighted by the corresponding quality factor and the corresponding output.

[0115] The computer-readable storage medium (404) of any one of items 19 and 15 to 18, storing therein further computer-readable program code (406), which, in response to execution by the processing circuit (402), causes the computer system (400) to further at least:

[0116] executing a third software application, the third software application being the second certifiable or qualified software application, when the certifiable software application and the qualified software application are executed, the third software application being executed to process the input data to produce a second corresponding output,

[0117] wherein causing the computer system to review (306) the output includes causing the output from the certifiable software application to be reviewed by comparing it with the corresponding output and a second corresponding output, and

[0118] Wherein causing the computer system to generate (308) an alert includes causing the computer system to generate an alert that the output from the certifiable software is not verified when the comparison indicates that the output differs from both the corresponding output and the second corresponding output by more than a threshold.

[0119] The computer-readable storage medium (404) of any one of items 20 and 15 to 19, wherein the computer system (400) is an avionics system of an aircraft moving toward a hub defining an aircraft target, the certifiable software application is a certifiable object detection application, and the qualified software application is a qualified object detection application,

[0120] wherein causing the computer system to execute (302) the authenticatable software application comprises causing the computer system to execute the authenticatable object detection application to process the image of the hub to produce an output in which an object is detected in the image,

[0121] wherein causing the computer system to execute (304) a qualified software application comprises causing the computer system to execute a qualified object detection application to process the image of the hub to produce a corresponding output in which an object is detected in the image, and

[0122] Wherein causing the computer system to review the output (306) includes causing the computer system to review the output by comparing it with a corresponding output to verify the output from the authenticatable object detection application.

[0123] Item 21. The computer-readable storage medium (404) of item 20, wherein the output from the certifiable object detection application is verified when the comparison indicates that the output differs from the corresponding output by no more than a threshold that decreases with the distance between the aircraft and the hub.

[0124] Benefiting from the teachings presented in the foregoing description and the associated drawings, many modifications and other embodiments of the disclosure described herein will be thought of by those skilled in the art to which the disclosure belongs. Therefore, it should be understood that the disclosure is not limited to the specific embodiments disclosed, and modifications and other embodiments are intended to be included within the scope of the appended claims. In addition, although the foregoing description and the associated drawings describe exemplary embodiments in the context of certain example combinations of elements and / or functions, it should be understood that different combinations of elements and / or functions can be provided by alternative embodiments without departing from the scope of the appended claims. In this regard, for example, as set forth in some of the appended claims, combinations of elements and / or functions different from the elements and / or functions explicitly described above are also considered. Although specific terms are employed herein, they are used only in a general and descriptive sense, and not for the purpose of limitation.

Claims

1. A computer system (400), comprising: a memory (404) configured to store computer readable program code (406); as well as The processing circuit (402) is configured to access the memory and execute the computer readable program code to cause the computer system to at least: executing (302) a certifiable software application consisting of static program instructions that are sequentially executed to process input data to produce output, the computer system being configured to operate on the output, the certifiable software application being certifiable for the computer system according to a certification standard; executing ( 304 ) a qualified software application during execution of the certifiable software application, the qualified software application processing the input data using a model iteratively constructed by a machine learning algorithm to produce a corresponding output, the qualified software application being non-certifiable for the computer system according to a certification standard; reviewing the output from the certifiable software application by comparing it to the corresponding output from the qualified software application (306) to verify the output from the certifiable software application to improve the integrity of the computer system; as well as When the comparison indicates that the output from the certifiable software application differs from the corresponding output by more than a threshold, an alert is generated (308) that the output from the certifiable software application is not verified.

2. The computer system (400) according to claim 1, wherein: When the comparison indicates that the output from the certifiable software application does not differ from the corresponding output from the qualified software application by more than the threshold, the output from the certifiable software application is verified, and the memory (404) stores further computer readable program code (406), the processing circuit (402) being configured to execute the further computer readable program code (406) to cause the computer system to further at least: An operation is performed on the output from the certifiable software application and the corresponding output from the qualified software application is excluded (310).

3. A computer system (400) according to any preceding claim, wherein: When the comparison indicates that the output from the certifiable software application does not differ from the corresponding output from the qualified software application by more than the threshold, the output from the certifiable software application is verified, and the memory (404) stores further computer-readable program code (406) that the processing circuit (402) is configured to execute to cause the computer system to further perform at least the following operations: generating a combined output from the output from the certifiable software application and the corresponding output; and An operation is performed on the combined output.

4. The computer system (400) according to claim 3, wherein: causing the computer system to execute (302) the certifiable software application and to execute (304) the qualified software application, further comprising causing the computer system to generate respective figures of merit for the output from the certifiable software application and the corresponding output, and wherein causing the computer system to generate the combined output comprises causing the computer system to perform a Kalman filter on the output from the certifiable software application and the corresponding output weighted by the respective quality factors.

5. The computer system (400) according to claim 2 or 3, wherein: The memory (404) stores further computer readable program code (406) that the processing circuit (402) is configured to execute to cause the computer system to further perform at least the following operations: executing a third software application, the third software application being a second certifiable or qualified software application, the third software application being executed to process the input data to produce a second corresponding output, when the certifiable software application and the qualified software application are executed, wherein causing the computer system to review (306) the output from the certifiable software application comprises causing the computer system to review the output from the certifiable software application by comparing the corresponding output and the second corresponding output, and Wherein causing the computer system to generate (308) the alert includes causing the computer system to generate the alert that the output from the certifiable software application is not verified when the comparison indicates that the output from the certifiable software application differs from both the corresponding output and the second corresponding output by more than the threshold.

6. The computer system (400) according to claim 2 or 3, wherein: the computer system is an avionics system of an aircraft moving toward a hub defining an aircraft target, the certifiable software application is a certifiable object detection application, and the qualified software application is a qualified object detection application, wherein causing the computer system to execute (302) the authenticatable software application comprises causing the computer system to execute the authenticatable object detection application to process an image of the hub to produce an output in which the object is detected in the image, wherein causing the computer system to execute (304) the qualified software application comprises causing the computer system to execute the qualified object detection application to process the image of the hub to generate a corresponding output that the target is detected in the image, and Wherein causing the computer system to review the output from the certifiable software application (306) includes causing the computer system to review the output from the certifiable software application by comparing it with the corresponding output to verify the output from the certifiable object detection application.

7. A method (300) of improving the integrity of a computer system (400), the method comprising: the computer system: executing (302) a certifiable software application consisting of static program instructions that are sequentially executed to process input data to produce output, the computer system being configured to operate on the output, the certifiable software application being certifiable for the computer system according to a certification standard; executing ( 304 ) a qualified software application during execution of the certifiable software application, the qualified software application processing the input data using a model iteratively constructed by a machine learning algorithm to produce a corresponding output, the qualified software application being non-certifiable for the computer system according to a certification standard; reviewing (306) the output from the certifiable software application by comparing it to the corresponding output from the qualified software application to verify the output from the certifiable software application to improve the integrity of the computer system; and When the comparison indicates that the output from the certifiable software application differs from the corresponding output by more than a threshold, an alert is generated (308) that the output from the certifiable software application is not verified.

8. The method (300) according to claim 7, wherein: When the comparison indicates that the output from the certifiable software application does not differ from the corresponding output from the qualified software application by more than the threshold, the output from the certifiable software application is verified, and the method further includes, the computer system: An operation is performed on the output from the certifiable software application and the corresponding output from the qualified software application is excluded (310).

9. The method (300) according to any one of claims 7 to 8, wherein: When the comparison indicates that the output from the certifiable software application does not differ from the corresponding output from the qualified software application by more than the threshold, the output from the certifiable software application is verified, and the method further includes, the computer system: generating a combined output from said output from said certifiable software application and said corresponding output; as well as An operation is performed on the combined output.

10. The method (300) according to claim 9, wherein: executing (302) the certifiable software application, and executing (304) the qualified software application, further comprising generating respective figures of merit for the output from the certifiable software application and the corresponding output, and wherein generating the combined output comprises Kalman filtering the output from the certifiable software application and the corresponding output weighted by the respective quality factors.

11. The method (300) according to any one of claims 7 to 8, further comprising: executing a third software application, the third software application being a second certifiable or qualified software application, the third software application being executed to process the input data to produce a second corresponding output, when the certifiable software application and the qualified software application are executed, wherein reviewing (306) the output from the certifiable software application comprises reviewing the output from the certifiable software application by comparing the corresponding output and the second corresponding output, and Wherein generating (308) the alert comprises generating the alert that the output from the certifiable software application is not verified when the comparing indicates that the output from the certifiable software application differs from both the corresponding output and the second corresponding output by greater than the threshold.

12. The method (300) according to any one of claims 7 to 8, wherein: the computer system is an avionics system of an aircraft moving toward a hub defining an aircraft target, the certifiable software application is a certifiable object detection application, and the qualified software application is a qualified object detection application, wherein executing (302) the authenticatable software application comprises executing the authenticatable object detection application to process an image of the hub to produce an output in which the object is detected in the image, wherein executing (304) the qualified software application comprises executing the qualified object detection application to process the image of the hub to produce a corresponding output that the target is detected in the image, and Wherein, reviewing (306) the output from the certifiable software application includes reviewing the output from the certifiable software application by comparing it with the corresponding output to verify the output from the certifiable object detection application.

Images