Supervisory device configured to communicate with an onboard peripheral computing device on board an aircraft and associated electronic software application execution system

Abstract

Supervisory device configured to communicate with an onboard peripheral computing device and associated electronic software application execution system. This supervisory device is configured to communicate with an onboard peripheral computing device, said onboard peripheral device being configured to execute software applications in connection with one or more certified operating onboard electronic devices, the supervisory device comprising a human-machine interface and being configured to receive, from said peripheral computing device, observability signals including data relating to local computing resources of the onboard peripheral computing device and information relating to the execution of at least one of said software applications.the monitoring device (82) comprising at least one computing processor configured to calculate (90) an operating state of said peripheral computing device and an execution state of said at least one software application from the received observability signals and to display (92) said operating state and said execution state on said human-machine interface. Figure for the abbreviation: Figure 4,

Description

Title of the invention: Supervision device configured to communicate with a peripheral computing device on board an aircraft and associated electronic system for executing software applications

[0001] The present invention relates to a supervision device configured to communicate with a peripheral computing device on board an aircraft, as well as an associated electronic system for executing software applications.

[0002] The invention lies in the field of avionics, and more particularly in the field of executing software applications on board an aircraft.

[0003] The invention applies to any type of aircraft, with or without a pilot on board, in both a civil and military context.

[0004] As is known, aircraft include on-board electronic devices (or on-board computers) with certified operation according to an avionics certification standard. In particular, any software application embedded in this type of device must be compliant and certified. Thus, the on-board software applications, which are useful for navigation, for controlling on-board equipment, for example various sensors, e.g. optical or thermal cameras, radars, etc., are limited in number, because they are expensive to develop and certify. In addition, the computing resources (e.g. CPU, ROM, RAM) of certified on-board computers are generally pre-allocated, to guarantee security and limit any risk of hacking or external intrusion. Thus, the hardware and software structure of on-board computers with certified operation is rigid, any evolution or modification being slow and expensive.

[0005] However, the need for computing resources, e.g. processors and memories, for aeronautical operations is constantly increasing, with increasingly sophisticated software applications being able to be advantageously implemented. In addition, new uses are developing with the advent of new types of lightweight aircraft, for example drones or urban taxi aircraft, allowing in particular the use of on-board sensors for new data collection missions, for surveillance applications, maintenance, etc.

[0006] There is therefore a need to increase the computing capacity of on-board devices to enable the execution of software applications linked to these new needs.

[0007] Systems for offloading software application executions to ground computing centers have been proposed. However, to the extent that commands of on-board equipment and data retrievals provided by such equipment are necessary, the remoteness of applications to the ground is not always possible, depending on the needs, particularly in terms of bandwidth and latency, and, where applicable, in terms of reliability / integrity of calculations and data protection.

[0008] There is a need to execute software applications in connection with equipment on board an aircraft, which combines the increase in available computing resources and reliability.

[0009] Furthermore, it is desirable to provide means for supervising the operation of on-board computing devices.

[0010] To this end, the invention proposes, according to one aspect, a supervision device configured to communicate with a peripheral computing device on board an aircraft, said onboard peripheral device being configured to execute software applications in connection with one or more onboard electronic devices with certified operation, the supervision device comprising a human-machine interface and being configured to receive observability signals from said peripheral computing device, said observability signals comprising data relating to local computing resources of the onboard peripheral computing device and information relating to the execution of at least one of said software applications. The supervision device comprises at least one computing processor configured to

[0011] -a) calculating an operating state of said peripheral computing device and a execution status of said at least one software application from the received observability signals and

[0012] -b) displaying said operating state and said execution state on said interface man machine.

[0013] Advantageously, the supervision device allows an operator to follow, in real time or in delayed time, the progress of a mission of execution of a plurality of software applications by an aircraft, for example from a base located on the ground. This makes it possible to detect possible operating problems and to improve or apply predictive maintenance for the onboard peripheral computing device and / or the onboard electronic devices with certified operation.

[0014] The supervision device according to the invention may also have one or more of the characteristics below, taken independently or in any technically conceivable combination.

[0015] The supervision device is configured, for a mission of executing a plurality of software applications on a trajectory of the aircraft, to display on the human-machine interface a plurality of trajectory portions, each portion of trajectory corresponding to an operating state of the peripheral computing device and / or on-board electronic devices.

[0016] The monitoring device is further configured to display information relating to each portion of trajectory, indicating said operating state and a root cause of the operating state.

[0017] The root cause is received from the on-board peripheral device or determined, by calculation, by said supervisory device, based on said observability signals and additional information including environmental information and connectivity information.

[0018] The supervision device is configured to perform the calculation a) and the display b) either substantially in real time, upon receipt of said observability signals from the on-board calculation device, or subsequently.

[0019] The supervision device is configured to implement a digital twin of the aircraft.

[0020] According to another aspect, the invention relates to an electronic system for executing software applications comprising a computing system on board an aircraft, and at least one remote computing system located on the ground, the onboard computing system comprising at least one on-board electronic device with operation certified according to an avionics certification standard, and a peripheral computing device, the remote computing system comprising a supervision device as briefly described above.

[0021] The electronic system for executing software applications may also have one or more of the characteristics below, taken independently or in any technically conceivable combination.

[0022] The system is such that the on-board peripheral computing device comprises a first communication interface configured to communicate, according to a first communication protocol, with one or more on-board electronic devices with certified operation, a second radio communication interface, configured to communicate, according to a second radio communication protocol, with the remote computing system located on the ground, the peripheral computing device comprising at least one electronic memory unit and at least one computing processor forming local computing resources, the computing processor being configured to implement:

[0023] - a module for managing a plurality of software applications stored in said electronic memory, each software application being adapted to provide at least one control parameter and / or receive at least one piece of data from at least one on-board electronic device with certified operation;

[0024] - a module for activating the execution of at least one of said software applications stored, configured to receive at least one request to execute at least one software application from an external control device and / or to validate a condition for activating the execution of at least one software application;

[0025] - an execution stack management module, configured to determine, for each software application to be executed, an allocation of local computing resources of said peripheral computing device to execute all or part of said software application to be executed, and to launch the local execution of said part of the application to be executed and to request, in the case where a first part of said application is executed locally, a remote execution, by the remote computing system, of a second part of the application to be executed, complementary to the first part.

[0026] The supervision device is configured to display, following the determination of the execution, for a software application, in two parts, respectively a first part executed by the embedded peripheral computing device and a second part executed by the remote computing system, to display information relating to said first and second parts.

[0027] The monitoring device is further configured to display a root cause of the two-part execution determination.

[0028] Other characteristics and advantages of the invention will emerge from the description given below, for information purposes only and in no way limiting, with reference to the appended figures, among which:

[0029] [Fig-1] [Fig.l] illustrates an electronic system for executing lo applications software according to one embodiment;

[0030] [Fig.2] [Fig.2] illustrates a method of executing software applications according to a first embodiment;

[0031] [Fig.3] [Fig.3] illustrates a method of executing software applications according to a second embodiment;

[0032] [Fig.4] [Fig.4] is a schematic representation of the main functional blocks tionals of a supervision device according to one embodiment;

[0033] [Fig.5] [Fig.5] is an example of a display on the human-machine interface of a device for supervising portions of trajectories relating to a mission of executing a plurality of software applications by an aircraft.

[0034] The electronic system for executing software applications, on board an aircraft, finds applications in all types of aircraft, and more particularly for lightweight aircraft such as drones, carrying equipment such as image sensors, such as optical cameras, thermal cameras, imaging radars, positioning sensors (GNSS, inertial units, anemo-barometric sensors, radio navigation beacons, triangulation), measurement systems such as rangefinders, weather, traffic or terrain monitoring sensors and systems, radio or digital communication systems, radio frequency (RF) antennas, etc.

[0035] [Fig.l] schematically illustrates a system 2 which is an electronic system for executing embedded software.

[0036] System 2 is a system allowing on-board-ground cooperation.

[0037] The system 2 comprises on the one hand an on-board computing system 4, comprising devices on board an aircraft 6, and on the other hand, one or more ground computing systems 8.

[0038] The on-board computing system 4 comprises a set 10 of on-board electronic devices with certified operation according to an avionics certification standard, comprising at least one on-board computer 12, and on-board equipment 14, e.g. sensors, for example cameras, radars, actuator, navigation system, automatic pilot system, flight controls, geolocation system, certified surveillance systems (terrain traffic, weather), certified communication systems (e.g. radios).

[0039] For example, certified operating on-board devices comply with one or more of the following standards: RTCA DO178B for the software certification level and RTCA DO 160 for the hardware certification part, AEEC Arinc 702A for the flight management system, etc.

[0040] In one embodiment, the on-board computer 12 is configured to implement certified software applications, for example applications for assisting navigation, controlling the operation of the sensors 14 and receiving and processing data from the sensors 14.

[0041] The on-board computer 12 comprises hardware resources, in particular a central processing unit comprising in particular one or more processors 15, adapted to execute computer program instructions when the processing unit is powered up. The on-board computer 12 also comprises information storage units 16, comprising in particular random access memory (RAM) units, ROM type memory units, adapted to store data and executable code instructions, a communication interface 18 and a human-machine interface 17.

[0042] In addition, the on-board computing system 4 advantageously comprises an on-board peripheral computing device 20.

[0043] This peripheral computing device 20 advantageously makes it possible to increase the on-board computing capacity, in a more flexible manner, allowing easier and less expensive software and hardware updates than for the on-board computer 12.

[0044] The peripheral computing device 20 comprises a first communication interface 22 configured to communicate, according to a first communication protocol, with the on-board electronic device(s) with certified operation, and in particular with the on-board computer 12.

[0045] In particular, the first communication interface 22 is configured to communicate, according to a bidirectional communication, with the communication interface 18 of the on-board computer 12.

[0046] For example, the communication between the peripheral computing device 20 and the on-board computer 12 is a wired communication. Alternatively, the communication between the peripheral computing device 20 and the on-board computer 12 is a wireless communication, for example of the Wifi or Bluetooth type.

[0047] Furthermore, the peripheral computing device 20 comprises a second radio communication interface 24, configured to communicate, according to a second radio communication protocol, with the remote computing system(s) 8 located on the ground.

[0048] The second communication protocol is a radio communication protocol, for example a standard 4G or 5G mobile telephone protocol, the SATCOM satellite communication protocol, the Datalink AOC / ATC protocol (for “Aeronautical Operation Control” / “Air Traffic Control”).

[0049] According to variants, the peripheral computing device 20 comprises several second radio communication interfaces 24, according to several radio communication protocols, making it possible to communicate with external devices, by a second radio communication interface 24A making it possible to communicate with the computing system 8 and a second radio communication interface 24B making it possible to communicate according to another radio communication protocol, for example with one or more external control devices 25, configured to send commands for implementing software applications by the peripheral computing device 20 and to receive data, in particular data from on-board sensors, in return.

[0050] An external control device 25 is for example a tablet, a laptop, a smartphone (or ordiphone).

[0051] Such an external control device 25 is for example also on board the aircraft.

[0052] In one embodiment, such an external control device 25 is operated by an operator on board the aircraft, for example a pilot of the aircraft, via a human-machine interface, e.g. a touch screen or any other suitable type of human-machine interface (not shown here).

[0053] Thus, the peripheral computing device 20 is configured to communicate, bidirectionally, with the remote computing system 8 and with at least one external control device 25.

[0054] The peripheral computing device 20 further comprises a processing unit 26, comprising one or more processors (CPU or GPU), and at least one electronic memory unit 28, these elements being adapted to communicate via a communication bus (not shown).

[0055] In the example of [Fig.l] only one processing unit 26 and one electronic memory unit 28 are shown.

[0056] The electronic memory unit 28 includes in particular memories of the RAM, ROM type, any type of non-volatile memory (for example EPROM, EEPROM, FLASH, NVRAM).

[0057] The set of processing unit(s) 26 and electronic memory unit(s) 28 forms the local computing resources of the peripheral computing device 20.

[0058] Preferably, the peripheral computing device 20 is modular, with several processing and memory units being able to be added, which allows a dynamic increase in the available resources, depending on needs.

[0059] The dynamic increase in the available resources is limited, in practice, in certain embodiments, by the electrical energy consumption and / or the total weight of such a peripheral computing device 20. Indeed, the total weight may be limited for certain types of aircraft, for example for drones, depending on the aerodynamic lift of the drone, and / or its maneuverability. Similarly, the electrical energy consumption when the peripheral computing device executes software applications with maximum occupation of the computing and storage resources 26, 28, must be taken into account, in order to ensure the completion of a planned mission, over a planned mission duration.

[0060] Here, mission is understood to mean a set of tasks to be carried out, for example by on-board sensors, in connection with associated flight indications (trajectories, durations, waypoints, etc.).

[0061] The electronic memory unit 28 is configured in particular to store a library of software applications 35, comprising a variable number of software applications APPi, APP2... APPi.

[0062] Each software application is stored in the form of executable program code. The term "executable application" is also used to refer to such a software application.

[0063] For example, the software applications are obtained by downloading (in English “Dataloading”), from an external control device 25 or from a ground computing system 8.

[0064] Alternatively or additionally, a software application APP may be obtained from a computer-readable medium, for example a medium capable of storing the electronic instructions and to be coupled to a bus of a computer system (e.g. USB key, non-volatile memory (e.g. EPROM, EEPROM, FLASH, NVRAM), a magnetic card or an optical card).

[0065] In addition, the electronic memory unit 28 is also configured to store data and results 37, including data obtained from the certified operating on-board electronic devices (e.g., images obtained by on-board sensors, operating data, data relating to the trajectory or performance of the aircraft), and results obtained by the execution of the software applications APP;. These data and results 37 can be transmitted to the ground as “observability signals”, allowing a human operator or a system to monitor the proper functioning of the overall system, ensure debugging, and perform simulations or mission replay.

[0066] Furthermore, the electronic memory unit 28 is configured to store parameters useful for implementing a method for executing software applications as described below, in particular parameters relating to the allocation of computing resources per software application, where appropriate, a table of execution priorities or a set of static rules defining priorities for executing software applications.

[0067] The processing unit 26 is configured to execute:

[0068] - a module 30 for managing a plurality of software applications, the applications software being stored in the electronic memory unit 28, in particular in an application library 35, each software application being adapted to provide at least one control parameter and / or receive at least one piece of data from at least one on-board electronic device with certified operation;

[0069] - a module 32 for activating the execution of at least one of said applications lo stored software, configured to receive at least one request for execution of at least one executable application from an external control device and / or to validate a condition for activating execution of at least one software application,

[0070] -a module 34 for managing an execution stack, configured to determine, for each software application to be executed, an allocation of local computing resources 26, 28 of the peripheral computing device 20 to execute all or part of the application to be executed, and to launch the local execution of said part of the application to be executed and to request, in the case where only a first part of said application is executed locally, a remote execution, by the remote computing system 8, of a second part of the application to be executed, complementary to the first part;

[0071] - a data and results management module 36, configured to consolidate and store the results, and communicate the results to the external control device and / or the remote computing system.

[0072] The modules 30, 32, 34, 36 are adapted to cooperate, as described in more detail below, to implement a method of executing software applications as described in more detail below.

[0073] In one embodiment, the modules 30, 32, 34, 36 are produced in the form of software instructions forming a computer program, which, when executed by a computer, implements a method for executing software applications according to the invention.

[0074] The computer program comprising software instructions is further capable of being recorded on a non-transitory computer-readable medium. The computer-readable medium is, for example, a medium capable of storing the electronic instructions and of being coupled to a bus of a computer system. For example, the readable medium is an optical disk, a magneto-optical disk, a ROM memory, a RAM memory, any type of non-volatile memory (for example EPROM, EEPROM, FLASH, NVRAM), a magnetic card or an optical card.

[0075] In a variant not shown, the modules 30, 32, 34, 36 are each produced in the form of programmable logic components, such as FPGAs (Field Programmable Gate Arrays) of microprocessors, GPGPU components (General-purpose processing on graphics processing), or even dedicated integrated circuits, such as ASICs (Application Specific Integrated Circuits)•

[0076] The peripheral computing device 20 is thus configured to locally execute a first part of the applications to be executed, and to delegate, if necessary, a second part of execution to the remote computing system 8. Indeed, in certain cases, the first part is equal to 100% of the software application to be executed, in which case there is no need to delegate a second part of execution to the remote computing system.

[0077] In one embodiment, the remote computing system 8 comprises one or more interconnected electronic computing devices.

[0078] Alternatively, the remote computing system 8 comprises or is connected to a cloud of computers or “cloud” in English.

[0079] In a functional representation, the remote calculation system 8 comprises a processing unit 40, comprising one or more calculation processors, an electronic memory unit 42, a ground communication interface 44, adapted to communicate with the second communication interface 24 of the on-board peripheral calculation device 20, and a human-machine interface 46.

[0080] The processing unit 40 is configured to execute parts of the applications lo APPi software, by executing a corresponding part of the executable code, stored in the electronic memory unit 42, upon request from the peripheral computing device 20, to store data and calculation results and to transmit these calculation results, via the communication interface 44, to the peripheral computing device 20 and / or to an external control device 25.

[0081] According to a variant, the remote calculation system 8 located on the ground is also configured to transmit calculation data and results to another aircraft 6, for example in the context of a mission involving a plurality of aircraft.

[0082] Advantageously, such execution stack management is dynamic and flexible, and makes it possible to take into account one or more criteria from among: a criterion of availability of computing resources, a connectivity criterion, a criterion of criticality of data, a priority criterion, as explained in more detail below.

[0083] [Fig.2] is a synopsis of the main steps of a method for executing software applications, implemented in the electronic system 2 for executing software applications, in a first embodiment.

[0084] As will become more clear from the description to follow, the method comprises a set of steps implemented by a processing unit, comprising at least one processor, of an on-board peripheral computing device 20. These steps are executed after powering up the peripheral computing device.

[0085] The method comprises an initialization step 50, which is a preliminary step, executed at a later time than the other steps of the method.

[0086] During the initialization step, in particular, the first and second communication interfaces are initialized, in particular configured with respective communication addresses, so as to allow communications, on the one hand with the on-board electronic devices 10 with certified operation, on the other hand with the external devices 25, and with the ground computing system 8.

[0087] Alternatively, the peripheral computing device executes the initialization of the communication interfaces following a request, for example transmitted by an operator or by an external application via an external control device 25.

[0088] Furthermore, in the initialization step 50, during the initialization of the first communication interface with the on-board equipment, first configuration structures, eg first configuration tables are received from at least one on-board computer 12, and stored in a local electronic memory unit of the peripheral computing device.

[0089] Similarly, during the initialization of the second communication interface with the remote computing system, second configuration structures, e.g. second configuration tables are received from at least one computing system. remote 8, and stored in a local electronic memory unit of the peripheral computing device.

[0090] The first and second configuration structures contain in particular connectivity information (eg identifiers and communication addresses).

[0091] Optionally, during the initialization step, the APPi software application library is re-initialized.

[0092] Optionally, the data and results 37 stored in the memory unit of the peripheral computing device are erased during initialization 50.

[0093] According to variants, the APP applications; and / or the data and results stored during prior use of the peripheral computing device are retained.

[0094] Furthermore, during the initialization phase, in one embodiment, parameters useful for implementing the method for executing software applications are obtained and stored. These parameters include in particular parameters relating to the allocation of resources per software application, for example for a minimum allocation per software application, a maximum allocation or an average allocation, and / or a table of execution priorities or a set of static rules defining priorities for executing software applications.

[0095] The resource allocation parameters contain for example: a computing power, for example expressed in processor clock frequency (e.g. in GHz), and / or an associated storage capacity, expressed for example in kilobytes, and / or a percentage of bandwidth for first communication with the on-board electronic devices with certified operation, and / or a percentage of bandwidth for second communication with the remote computing system.

[0096] In one embodiment, the resource allocation parameters are grouped in the form of a parameter vector, containing a value for each of the parameters mentioned below:

[0097] - PI: allocated computing power;

[0098] - P2: allocated storage capacity;

[0099] - P3: percentage of first communication bandwidth;

[0100] - P4: percentage of second communication bandwidth.

[0101] In variants, a vector comprising only a subpart of the allocation parameters is defined.

[0102] In another variant, only one of the parameters mentioned above is used.

[0103] The method comprises a step 52 of receiving a command to add / delete a software application, from an external control device, or from the ground computing system or from one of the on-board devices.

[0104] Preferably, a deletion command is received from an external control device or an authorized operator, for example the owner of the software application. to delete.

[0105] For example, authorized operator authentication is implemented by an authentication method, many variants of such authentication methods being known.

[0106] According to one embodiment, the addition / update of an application is carried out by an operator, preferably an authorized operator, by means of a data storage medium (e.g. USB key).

[0107] Following receipt of such a command, the method comprises a step 54 of application management, which updates the library of software applications based on the commands received.

[0108] Step 54 comprises the reception of a new software application or an update of a previously stored software application, by means of a wired link or to be downloaded by means of a radio communication link, then the storage of the new applications APPj, and / or the deletion of the applications APP; from the applications library.

[0109] The software applications stored in the application library 35 are adapted to provide at least one control parameter and / or receive at least one piece of data from at least one on-board electronic device.

[0110] As an optional addition, following the receipt of a new software application or an update of an existing software application, the method implements validations of the integrity and validity of the downloaded applications, by usual methods (authentication, validation of cyclic redundancy check (CRC), etc.).

[0111] Subsequently, the method comprises activating execution 56 of one or more of the stored software applications, which then become software applications to be executed.

[0112] Indeed, the parallel execution of several software applications is managed.

[0113] The execution activation is based either on the reception of a request to execute at least one software application from an external device, or on the validation of an execution activation condition.

[0114] In the case of validation of an execution activation condition, the activation is automatic.

[0115] For example, the condition for activating the execution of a software application APP; is the end of execution of a software application APPj or corresponds to a reduction in the resource requirement of the software application Appj compatible with its operation.

[0116] In addition, commands to pause or stop execution of one or more software applications may also be received (step 58), their management being carried out directly.

[0117] Stopping execution allows resources to be freed up and therefore available resources to be redistributed. As with execution, automatic validation of a stop condition also allows the execution of an application to be stopped or paused.

[0118] Steps 54, 56, 58 are preferably executed in the background, substantially in parallel, so as to constantly take into account the reception of new software applications or the updating of existing software applications, as well as execution activations and execution stop or pause commands.

[0119] The method further comprises a step 60 of managing an execution stack of the software applications to be executed. This step is repeated regularly, at predetermined time intervals or upon event (for example request to start a new application, event which terminates or significantly modifies the computing resource requirements of an application).

[0120] The management step 60 implements for each application to be executed APPk, an allocation of resources, in particular of computing resources (processor and memory) of the peripheral computing device, called local resources, to execute all or part of the application to be executed APPk.

[0121] For example, in one embodiment, the same quantity of local resources is allocated to each APPkj, the quantity being predetermined in the initialization step 50.

[0122] According to one variant, the resources are allocated dynamically, for example according to an expected response time per application, which makes it possible to dynamically define execution priorities.

[0123] Alternatively, resource allocation is performed based on priorities, defined in a previously recorded execution priority table or according to a previously programmed set of static rules.

[0124] According to one variant, the priorities are dynamically updated by means of an external control device.

[0125] Thus, the management module also implements a priority criterion.

[0126] For each software application to be executed APPk, depending on the resources necessary for its execution, it is possible to determine, where appropriate, a first part of the application APPk which is executed locally, and a second part APPk2, complementary to the first part, to be executed by the remote computing system on the ground.

[0127] In other words, local resources and remote resources are allocated for the execution of the same software application APPk.

[0128] Thus, a criterion of availability of computing resources is implemented to determine the local or remote execution of each software application.

[0129] In addition, the availability of a communication bandwidth with the ground computing system is also taken into consideration. Thus, a connectivity criterion is taken into account.

[0130] Indeed, for example for software applications that use high-speed data, for example video data, a significant bandwidth is necessary to carry out data processing by the remote computing system located on the ground. In addition, for an aircraft in motion, the communication conditions according to the second radio communication protocol may vary depending on the movement, weather conditions, etc.

[0131] Consider for example a drone for detecting and mapping obstacles such as electricity pylons, and whose mission consists of flying above a given geographical area, detecting obstacles of the “pylon” type, characterizing them (width, height), geo-referencing them (i.e. calculating their geographical position in a given reference frame), and deducing the mesh of electric cables which connects them.

[0132] Detection can be done using cameras embedded in the drone, the resolution, field width and frame rate of which will have a direct impact on the resource requirements: A video sequence in 4K definition / resolution, wide field, at 50 images / sec will, for example, require much more storage resources (RAM / ROM) and CPU (or GPU) resources for its processing, than a low resolution video sequence (480p for example), or with a lower field width or at 25 images / sec.

[0133] The power consumption, and consequently the autonomy of the drone, depends directly on the CPU / RAM / ROM load.

[0134] In the case of a distributed calculation between the local application APPk b and the application to be executed by the remote calculation system on the ground APPk 2, the electrical consumption of the drone will also be a function of the communication bandwidth between the edge and the ground, and of the quality of the network, which conditions the means of communication used and the power necessary to supply the transmission and reception antennas.

[0135] If the computing capacity (CPU / GPU) and / or storage (RAM / ROM) is sufficient to process a 4K video sequence (criterion 1), and if the power consumption is compatible with the flight duration objectives of the Drone (criterion 2), the system may choose to have all the mapping, detection and georeferencing calculations carried out locally by the APPk k application

[0136] If the computing capacity (CPU / GPU) and / or storage (RAM / ROM) is insufficient to process a 4K video sequence (criterion 1), and if the power consumption is compatible with the drone's flight duration objectives (criterion 2), the system may choose to have the mapping, detection and geo-referencing calculations carried out locally by the APPkj2i application by sending the images taken locally by the camera on the ground device.

[0137] If the computing capacity (CPU / GPU) and / or storage (RAM / ROM) is sufficient to process a 4K video sequence (criterion 1), but the power consumption is incompatible with the drone's flight duration objectives (criterion 2), the system can choose between several solutions: - If the power consumption is too high due to the processing of all the images on board (CPU / RAM / ROM), but the power consumption to send the images to the ground is not too high (for example in the case where the communication link is of good quality), the system can choose to have part of the mapping and detection and geo-referencing calculations carried out locally by the APPk 4 application and another part by the APPk2 application, by sending part of the images taken locally by the camera to the ground device. - If the power consumption is not too high due to the processing of all the images on board (CPU / RAM / ROM), but the power consumption to send the images to the ground is too high (for example in the case where the communication link is of good quality), the system can choose to have all the mapping, detection and geo-referencing calculations carried out locally by the APPk>i application. The images will not be sent to the ground. - If the power consumption is too high due to the processing of all the images on board (CPU / RAM / ROM), and the power consumption to send the images to the ground is also too high (for example in the case where the communication link is of poorer quality), the system may choose to switch to a lower image resolution, or to store the images for longer, while waiting to find a better bandwidth. - If the computing capacity (CPU / GPU) and / or storage (RAM / ROM) is insufficient to process a 4K video sequence (criterion 1), but the power consumption is compatible with the drone's flight duration objectives (criterion 2), the system may choose to have all the calculations carried out by the APPk2 application, by sending the images taken locally by the camera to the ground device. - Finally, if the computing capacity (CPU / GPU) and / or storage (RAM / ROM) is insufficient to process a 4K video sequence (criterion 1), and the power consumption is incompatible with the drone's flight duration objectives (criterion 2), switch to a lower resolution and repeat the steps above. - If there is no solution, even for the lowest possible resolution, the system may disable the APPk application and record an error message to the ground.

[0138] The first and second parts to be executed respectively locally / remotely are for example defined in terms of percentage of code to be executed.

[0139] In certain cases, the first part is equal to 100% of the application to be executed, in which case there is no need to transfer part of the execution to the ground computing system.

[0140] Preferably, the software applications are coded to be executed in parts.

[0141] In one embodiment, all or part of each software application is also previously stored in the electronic memory 42 of the remote calculation system 8, which facilitates the delegation of calculations.

[0142] In one embodiment, a data criticality criterion is implemented to determine the second part APPk2, to be executed on the ground, thus making it possible to avoid transmitting data or results considered sensitive or confidential to the ground computing system, via the second transmission interface.

[0143] Such data considered confidential, and therefore to be protected, are for example data relating to the performance of the aircraft, or confidential data of the client who owns the application to be executed (e.g. airline, institution, company).

[0144] In particular, this makes it possible to limit any transmission of data associated with a critical flight or mission phase.

[0145] In the case where a second part of the software application is to be executed by the remote computing system, the method comprises a remote execution request 62, comprising the transmission of associated information (e.g. execution point, data to be used, intermediate results).

[0146] In a variant, in the absence of prior storage in an electronic memory unit of the remote calculation system, the executable code forming the second part APPk 2, complementary to the first part, to be executed by the remote calculation system, is transmitted to the remote calculation system during step 62.

[0147] The remote computing system executes the second part APPk2 of software application (step 64), and transmits the results of the calculations to the embedded peripheral computing device (step 66).

[0148] The peripheral computing device receives the results in step 68.

[0149] In one embodiment, the results are consolidated at consolidation step 70.

[0150] For example, consolidation involves temporal and / or spatial synchronization results, and / or data fusion, for example fusion of inertial sensor data with positions calculated by software processing from camera data.

[0151] Following this consolidation step 70, the consolidated results are stored and / or transmitted (step 72) to the external control device, initiator of the execution of the software application.

[0152] [Fig.3] illustrates a second embodiment of the application execution method implemented in the application execution system 2.

[0153] In this second embodiment of the application execution method implemented in the application execution system 2, a supervision device, which is either one of the external control devices or a device of the remote computing system on the ground, implements real-time supervision of the operation of the on-board peripheral computing device.

[0154] Preferably, the supervision device is part of the remote computing system on the ground and comprises a human-machine interface, allowing the display of information intended for an operator and the reception of commands from the operator.

[0155] In this embodiment, in addition to all of the steps described previously and which are not re-described here, the method comprises supervision 80 implemented by the supervision device.

[0156] A schematic representation of a supervisory device 82 is provided in [Fig.4],

[0157] The supervision device 82 is a programmable electronic device, comprising a communication interface 84 adapted to communicate with the second communication interface 24 of the on-board peripheral computing device 20.

[0158] It also includes a human-machine interface 85, including in particular one or more display screens, touch-sensitive or not, and means (keyboard, mouse, etc.), allowing interaction with an operator.

[0159] The supervision device 82 also comprises a processing unit (eg one or more processors) 86 and an electronic memory unit 88. The units 84, 85, 86, 88 are adapted to communicate via an internal communication bus.

[0160] The processor 86 is configured to implement a calculation module 90, configured to calculate an operating state of said peripheral calculation device and an execution state of said at least one software application from observability signals received from the peripheral calculation device 6, and a module 92 for displaying, in graphic form, on the human-machine interface 85, the operating / execution states, in particular during a mission requiring the execution of software applications along a trajectory, or after the execution of the mission.

[0161] The modules 90, 92 are adapted to cooperate, as described in more detail below, to implement the supervision step 80 described in detail below.

[0162] In one embodiment, the modules 90, 92 are produced in the form of software instructions forming a computer program, which, when executed by a computer, implements the supervision method described.

[0163] The computer program comprising software instructions is further capable of being recorded on a non-transitory computer-readable medium. The computer-readable medium is, for example, a medium capable of storing the electronic instructions and of being coupled to a bus of a computer system. For example, the readable medium is an optical disk, a magneto-optical disk, a ROM memory, a RAM memory, any type of non-volatile memory (for example EPROM, EEPROM, FLASH, NVRAM), a magnetic card or an optical card.

[0164] In a variant not shown, the modules 90, 92 are each produced in the form of programmable logic components, such as FPGAs (Field Programmable Gate Arrays), microprocessors, GPGPU components (General-purpose graphics processing), or even dedicated integrated circuits, such as ASICs (Application Specific Integrated Circuits).

[0165] The supervision 80 is carried out continuously, the supervision device comprising a human-machine interface and being configured to receive observability signals from the peripheral computing device 20.

[0166] In this embodiment, observability signals of the peripheral computing device, sent as and when, comprise data relating to the local computing resources and information relating to the execution of at least one of the software applications. The observability signals include in particular information relating to the use of the local resources of the peripheral computing device and of the stack of applications to be executed / currently executed.

[0167] The supervision device 82 is configured to calculate an operating state of the peripheral computing device and an execution state of the or each software application from the observability signals received.

[0168] The supervision device 82 then implements, for example, displays on the human-machine interface 85, representative of the operating state of the peripheral computing device (state of local resources, connectivity) and the execution state of the applications, with optionally in addition the data and results of these applications. These displays follow, in one embodiment, substantially in real time, the operation of the peripheral computing device. This allows an operator to define execution priorities in real time, making it possible to prioritize the execution of certain software applications and to put other software applications on low priority or on pause, and / or to modify the resource allocations in real time.

[0169] Commands relating to the execution priorities of the software applications and / or modifications to the allocation of resources are then transmitted via the supervision device to the on-board peripheral computing device.

[0170] In one embodiment, the supervision extends to all on-board devices, including certified operating devices, thus enabling overall supervision of the use of on-board resources. Indeed, via the on-board peripheral computing device, data provided by on-board sensors or devices are also transmitted to the supervision device for display.

[0171] In addition, in one or other of the embodiments described above, it is provided to record, for example in execution reports (e.g. files, tables) or any other appropriate data structure, in an adequate format, a set of status and execution information of the peripheral computing device and of the on-board electronic devices with certified operation, as well as additional information including environmental information and connectivity information, over the entire duration of the aircraft mission.

[0172] In one embodiment, the set of information recorded in the report(s) includes the observability signals, also transmitted to the monitoring device as and when.

[0173] For example, the set of information includes all the states of the execution stack, and the computing / hardware resources of the peripheral computing device, for example the software applications executed, the description of the computing resources allocated, the data and results, as well as additional information such as: the meteorological situation, the availability of the communication interfaces, the bandwidth available for communications with the ground computing system, internal operating data of the systems, the energy consumption taken by each application, and by the other systems of the aircraft, the flight profile.

[0174] The execution report(s) containing this set of information are transmitted to the ground supervision device.

[0175] These execution reports can then be used to carry out a subsequent replay phase, executed on the ground by the supervision device. The supervision device is configured to display, in cooperation with graphical representation mechanisms on the human-machine interface 85, a complete progress of the mission (replay), in particular successive states of the peripheral computing device and, optionally, of the on-board electronic devices, which allows maintenance and debugging operations. Among other things, such a replay phase also allows recalculation of the parameters provided in the initialization step 50 such as the minimum, maximum or average allocation of the resources of the peripheral computing device. peripheral.

[0176] [Fig. 5] illustrates an example of display 85 on a human-machine interface of a supervision device, hereinafter called supervision HMI, within the framework of a mission carried out by a drone equipped with a peripheral computing device 20 as described above.

[0177] The illustrated example is based on the example of the mission of capturing images by the on-board sensors of a drone, of detecting and mapping obstacles such as electricity pylons.

[0178] In this example, the trajectory carried out by the drone is illustrated, and divided into several portions of trajectory of different operation, as well as information Infol, ..., Info5 relating to the operating states of the peripheral computing device, to the execution states of the software applications, and optionally additional connectivity and / or context information (e.g. meteorological) for each portion of trajectory.

[0179] The mission takes place between a starting point D and an arrival point A, which may be the same. The scales are, for example, in latitude (ordinate) and longitude (abscissa). The supervision HMI thus makes it possible to follow the evolution of the drone's trajectory, on which information obtained from other observability signals is superimposed.

[0180] Advantageously, the graphic aspect of the trajectory is a means of illustrating the events linked to the mission.

[0181] In the example illustrated, the trajectory referenced TD i in solid line, starting from the starting point D and up to the point PI, corresponds to a nominal operation in which the image capture is carried out entirely on board by the application APPk i: the drone has just started its mission, its batteries are full, the connectivity is good, the CPU capacity and storage memory (local resources of the onboard peripheral device) are correct and therefore allow high-resolution images to be taken and processed on board.

[0182] Between points PI and P2, a trajectory “hole” indicates that the information relating to this part of the trajectory could not be transmitted, in the absence of means of communication, for example in the event of loss of the GNSS satellite positioning signal. For example, a peripheral computing device received and recorded information on loss of the GNSS satellite positioning signal from one or more certified operating on-board electronic devices.

[0183] Preferably, Infol information is displayed, indicating the reason for the anomaly observed, for example in the form of a boxed message, indicating “satellite location lost”, and as an optional addition, a root cause, for example “GNSS Masked”, when the root cause has been determined.

[0184] The determination of the root cause can be done: - either directly by one or more on-board electronic devices with certified operation (for example the certified GNSS receiver on board the drone which notes the reception of fewer than 4 satellites, a sign of masking), - either by the peripheral computing device which has, for example, correlation rules, to correlate a simple loss message from one of the on-board electronic devices with certified operation, with a terrain elevation map database, and deduce a satellite masking therefrom, - or by the supervision device, which may also have such correlation rules or have a digital twin (in English "Digital Twin") of the drone, capable of simulating the behavior of the Drone in its synthetic environment on the ground and also deducing a masking therefrom.

[0185] At point P2, the GNSS satellite positioning signal is reestablished, for example the peripheral computing device has received and recorded information on correct reception of the GNSS satellite positioning signal from one or more on-board electronic devices with certified operation, and can therefore retransmit its position.

[0186] Between points P2 and P3, a portion of trajectory referenced T2j3 is illustrated in a thin line, indicating an operating mode for sharing calculations between the onboard peripheral computing device and the remote computing system, i.e. the execution of software applications, for example digital image processing, shared in two parts APPk i and APPk2, one carried out on board, the other on the ground. For example, the allocation of the execution of a part on the ground is due to the reaching of a threshold of use of the random access memory (RAM) of the peripheral computing device.

[0187] Preferably, information Info2 relating to the state of the peripheral computing device (eg “RAM saturation”) and / or to the sharing of execution between edge and ground is displayed in connection with the portion of trajectory T2 3

[0188] Between point P3 and point P4, the portion of trajectory referenced T3 4 is represented in the form of thick dotted lines, which indicates that the peripheral computing device has entered a degraded mode of computing the video sequences, and has, for example, chosen to store the sequences on board rather than sending them to the ground.

[0189] The reason for the anomaly can be explained on the supervision HMI in the form of Info3 information, for example in text form to indicate that the calculation is suspended, and also indicate the root cause of this degraded mode, for example a loss of connection between the communication interface of the on-board peripheral calculation device and the communication interface of the remote calculation device (eg "LTE Lost"). This mode corresponds to the case where the Drone cannot perform the calculations alone and must delegate part of them to the outside (from point P2), but cannot

[0190]

[0191]

[0192]

[0193]

[0194] communicate properly with the remote computing device. Between point P4 and point P5, the portion of trajectory referenced T4 5 is represented in a thick solid line, to indicate a return to nominal mode (identical to the mode between starting point D and point PI). Between point P5 and point P6, trajectory T5 6 is grayed out by a 2D zone which indicates an anomaly in the quality of the images independent of the computing and communication means. Preferably, the reason for the anomaly (here a drop in the quality of the images processed by the APPk i application) is indicated in the form of Info4 information, for example in text form, rather than the root cause of this anomaly: "low visibility" (or "Low Visibility" in English). To be able to determine this information to be displayed, several solutions are possible depending on the computing means on board and on the ground. The determination can be carried out: - either directly by one or more on-board electronic devices with certified operation (for example, temperature and humidity sensors when they are on board, and capable of determining a risk of cloudiness), - either by the peripheral computing device which has, for example, correlation rules, to correlate sensor data (temperature and / or humidity), with an on-board weather forecast model or received by data link, and to deduce that the drone is moving in cloud cover, - either by the supervision device which may also have such correlation rules or have a digital twin of the drone, capable of simulating the behavior of the drone in its synthetic environment on the ground and also deducing the presence of clouds on the trajectory followed by the drone. Between point P6 and the current position of the drone, materialized by a visual indicator 100 in the form of a triangle, a thin dotted line indicates that the mission of the drone (here obstacle detection and mapping) has been interrupted. Preferably, associated information Info5 is displayed, indicating for example textually the termination of the mission, as well as the root cause “user command” which indicates that a human operator has manually terminated the photography / obstacle detection and mapping application APPk. By means of the described application execution system, the on-board hardware resources of an aircraft (computational resources, memory) are dynamically and flexibly increased, and the implementation of client software applications is then possible, while ensuring execution security. Advantageously, an aircraft carrying a peripheral computing device as described can be used as a mobile means of capturing data, like satellites.

[0195] Advantageously, the presence of an on-board peripheral device makes it possible to perform calculations with complex algorithms, which would not be possible with certified on-board devices due to the limitations imposed by the certifications.

[0196] Advantageously, the flexibility introduced by the possibility of moving parts of the execution of the applications to the ground makes it possible to adapt to the constraints of the aircraft, in particular the constraints of on-board weight and electrical energy consumption.

[0197] Advantageously, the supervision device allows monitoring and supervision, in real time or in delayed time, which allows the improvement of the implementation of the on-board computing device by feedback from the operator, for example debugging as well as the judicious management of the on-board resources by intervention of an operator. Advantageously, commands relating to the implementation of judiciously chosen on-board software applications are then sent by the supervision device to the on-board peripheral computing device.

Claims

Claims

1. Supervision device configured to communicate with a peripheral computing device on board an aircraft, said onboard peripheral device being configured to execute software applications in connection with one or more onboard electronic devices (12, 14) with certified operation, the supervision device comprising a human-machine interface and being configured to receive observability signals from said peripheral computing device, said observability signals comprising data relating to local computing resources of the onboard peripheral computing device and information relating to the execution of at least one of said software applications,the supervision device (82) being characterized in that it comprises at least one calculation processor configured to -a) calculate an operating state of said peripheral calculation device and an execution state of said at least one software application from the observability signals received and -b) display said operating state and said execution state on said human-machine interface.,

2. Supervision device according to claim 1, configured, for a mission of executing a plurality of software applications on a trajectory of the aircraft, to display on the human-machine interface, a plurality of trajectory portions, each trajectory portion corresponding to an operating state of the peripheral computing device and / or of the on-board electronic devices.

3. A monitoring device according to claim 2, further configured to display information relating to each trajectory portion, indicating said operating state and a root cause of the operating state.

4. A monitoring device according to claim 2 wherein said root cause is received from the on-board peripheral device or determined, by calculation, by said monitoring device, based on said observability signals and additional information including environmental information and connectivity information.

5. Supervisory device according to one of claims 1 to 4, configured to carry out the calculation a) and the display b) either substantially in real time real, upon receipt of said observability signals from the on-board computing device, or subsequently.

6. Supervision device according to one of claims 1 to 5, configured to implement a digital twin of the aircraft.

7. Electronic system for executing software applications comprising an onboard computing system (4) on an aircraft (6), and at least one remote computing system (8) located on the ground, the onboard computing system (4) comprising at least one onboard electronic device (12) with operation certified according to an avionics certification standard, and a peripheral computing device (20), the remote computing system comprising a supervision device (82) according to one of claims 1 to 6.

8. System according to claim 7, in which the on-board peripheral computing device comprises a first communication interface (22) configured to communicate, according to a first communication protocol, with one or more on-board electronic devices (12, 14) with certified operation, a second radio communication interface (24, 24A, 24B), configured to communicate, according to a second radio communication protocol, with the remote computing system (8) located on the ground, the peripheral computing device comprising at least one electronic memory unit (28) and at least one calculation processor (26) forming local computing resources, the calculation processor (26) is configured to implement: - a module (30) for managing a plurality of software applications (APPi, APP2, APP3, APPi) stored in said electronic memory (26),each software application being adapted to provide at least one control parameter and / or receive at least one data item from at least one on-board electronic device (12, 14) with certified operation; - a module (32) for activating the execution of at least one of said stored software applications, configured to receive at least one request for execution of at least one software application from an external control device (25) and / or to validate a condition for activating the execution of at least one software application; - a module (34) for managing an execution stack, configured to determine, for each software application to be executed, an allocation of local computing resources of said peripheral computing device to execute all or part of said software application to be executed, and, to launch the local execution of said part of the application to be executed and to request, in the case where a first part of said application is executed locally, a remote execution, by the remote computing system (8), of a second part of the application to be executed, complementary to the first part.

9. System according to claim 8, in which the supervision device is configured to display, following the determination of the execution, for a software application, in two parts, respectively a first part executed by the embedded peripheral computing device and a second part executed by the remote computing system (8), to display information relating to said first and second parts.

10. The system of claim 9, wherein the monitoring device is further configured to display a root cause of the two-part execution determination.

Images