Method and system for automatic characterization of learning data acquired by an aircraft

Abstract

Method and system for automatic characterization of learning data acquired by an aircraft This method comprises an acquisition (64) of digital images by on-board sensors, a first processing phase (60) implemented by a peripheral computing platform on board the aircraft, and a second processing phase (62) implemented by a remote computing system. The first phase comprises a consistency (70) between at least a part of the acquired digital images and flight data obtained from on-board avionics instruments, a storage of spatial and / or temporal location information in association with said digital images.The second phase comprises a labeling (80) of objects contained in said digital images, a calculation of characterization information (84) associated with the objects as a function of the spatial and / or temporal location information and at least one piece of complementary information, and a storage (86), in a learning database, of said detected objects in association with said label and the characterization information. Figure for the abstract: Figure 2.

Description

Title of the invention: Method and system for automatic characterization of learning data acquired by an aircraft

[0001] The invention relates to a method for automatically characterizing learning data acquired by an aircraft. It also relates to a system for automatically characterizing learning data acquired by an associated aircraft and an associated computer program.

[0002] The invention also relates to a method and a device for using learning data acquired by an aircraft and characterized by a method for automatic characterization of learning data acquired by an aircraft.

[0003] The invention lies in the field of data acquisition by aircraft and their use for machine learning of parameters of artificial intelligence models, for example for applications of object detection and classification, terrain recognition.

[0004] In particular, the invention applies to aircraft flying at low altitude, such as for example helicopters, drones, surveillance aircraft.

[0005] The term low altitude generally refers to an altitude less than or equal to 900 meters above sea level or AMSL (“Above Mean Sea Level”), or 300 meters above ground level if the ground is at an altitude greater than 900 meters above sea level.

[0006] Recently, the use of artificial intelligence has grown considerably in many application areas. Artificial intelligence models are developed by machine learning. These models, whether neural networks, random forests or any other architecture, are parameterized models, and the parameters have values ​​that are learned in an automated machine learning phase. In particular, in the case of supervised machine learning, the training database includes so-called labeled (or annotated) training data, i.e. associated with a validated label indicating the result to be obtained, for example the classification, by applying the classification model.

[0007] It is known in the field of artificial intelligence that the quality and quantity of training data are important to obtain a reliable and robust model.

[0008] Conventionally, prediction or classification models trained by machine learning (or prediction or classification methods by machine learning) are qualitatively characterized by precision and recall. Precision indicates the proportion of relevant items assigned to a given class i (true positives) among the set of items assigned to class i. Recall is the proportion of items relevant items proposed by the model from among the set of relevant items. A model is reliable when it has a high precision and recall rate. A model is robust if it is able to obtain a correct result from noisy input data.

[0009] When the amount of training data is small or when the training data is unbalanced, i.e. some classes are underrepresented, it is difficult to obtain a reliable and robust model.

[0010] It is then critical to characterize the learning data used for machine learning of parameters of an artificial intelligence model.

[0011] By characterizing is meant enriching each training data with additional information called characterization information, making it possible to evaluate the cases in which the model is likely to provide good or poor performance. As an explanatory example, if the training data are digital images acquired at more than 95% in sunny conditions and in less than 5% in rain / snow conditions, the classification model trained on this training data will be much more efficient when it is applied, in real time, to digital images acquired in sunny conditions, or even risk providing erroneous results on digital images acquired in rain / snow conditions.

[0012] The characterization of the learning data, in particular the learning data from digital images acquired by aircraft, makes it possible on the one hand to better determine in which practical applications the model will be efficient, and to direct the task of collecting learning data for a rebalancing

[0013] In the state of the art, we know the manual characterization of learning data, which is particularly laborious and time-consuming, or the analysis of learning data by artificial intelligence / machine learning models, which again poses problems of reliability of these analysis models.

[0014] It has also been proposed to generate synthetic training data, but such techniques are likely to introduce errors as well.

[0015] The invention aims to remedy the aforementioned drawbacks of the prior art, in the case of learning data obtained from digital images acquired by an aircraft.

[0016] To this end, the invention relates, according to one aspect, to a method for automatically characterizing learning data acquired by an aircraft and used for machine learning of parameters of an artificial intelligence model, the method comprising an acquisition of digital images by sensors on board said aircraft. The method comprises a first processing phase implemented by a peripheral computing platform, on board the aircraft, said peripheral computing platform comprising a communication interface configured to communicate with a remote computing system comprising a plurality of interconnected computing servers, and a second processing phase implemented by said remote computing system.

[0017] The first processing phase comprises a consistency between at least part of the acquired digital images and flight data obtained from on-board avionics instruments, a storage of associated spatial and / or temporal location information, in association with said digital images, and a transmission of said digital images and associated spatial and / or temporal location information to the remote computing system,

[0018] The second phase of treatment includes:

[0019] - a labeling of objects contained in said digital images, making it possible to obtain a classification label for each object from among a plurality of predetermined classes;

[0020] - a calculation of characterization information associated with the objects as a function of the spatial and / or temporal location information and at least one piece of complementary information, and

[0021] - a storage, in a learning database, of said detected objects in association with the label and at least one characterization information.

[0022] Advantageously, the implementation of the method for automatic characterization of learning data in two processing phases, a first processing phase by a peripheral computing platform, on board the aircraft, and a second processing phase by a remote computing system comprising a plurality of interconnected computing servers, makes it possible to correlate spatial and / or temporal acquisition location information with one or more complementary information, so as to automatically calculate characterization information. Thus, the method is automated and reliable.

[0023] The method for automatically characterizing learning data according to the invention may also have one or more of the characteristics below, taken independently or in any technically conceivable combination.

[0024] The method comprises, in the first processing phase, a storage of temporal information associated with each digital image, said temporal information being a time instant of acquisition of the digital image.

[0025] The flight data are time-stamped, and the consistency includes, for each processed digital image, a calculation of a geo-reference spatial position of said digital image as a function of the flight data acquired at the time instant of acquisition of the digital image.

[0026] The second processing phase includes a calculation of at least one geo-referenced position associated with each detected object.

[0027] The step of labeling objects comprises a detection, in each processed digital image, of one or more objects and tracing of a polygon circumscribed to each detected object, said at least one geo-referenced position being a position of a point of said polygon.

[0028] The polygon is divided into squares, a point is associated with each square, the object being associated with the geo-referenced positions of each of said points associated with the squares.

[0029] Said at least one additional information is obtained from at least one external data record.

[0030] Said at least one external record is a meteorological database and / or a geographical database. According to another aspect, the invention relates to a system for automatically characterizing learning data acquired by an aircraft and used for machine learning of parameters of an artificial intelligence model, the system comprising an aircraft equipped with on-board sensors configured to acquire digital images, the aircraft comprising a peripheral computing platform, the characterization system further comprising a remote computing system comprising a plurality of interconnected computing servers, the peripheral computing platform comprising a communication interface configured to communicate with the remote computing system, the peripheral computing platform comprising, for the implementation of a first processing phase,

[0031] a module for ensuring consistency between at least part of the acquired digital images and flight data obtained from on-board avionics instruments, for storing associated spatial and / or temporal location information, in association with said digital images, and for transmitting said digital images and associated spatial and / or temporal location information to the remote computing system,

[0032] the remote calculation system comprising, for the implementation of a second processing phase:

[0033] a module for labeling objects contained in said digital images, making it possible to obtain a classification label for each object from among a plurality of predetermined classes;

[0034] a module for calculating characterization information associated with the objects as a function of the spatial and / or temporal location information and at least one piece of complementary information, and

[0035] a module for storing, in a learning database, said detected objects in association with said label and at least one characterization information.

[0036] The invention also relates to a computer program implemented in a system for automatic characterization of learning data, comprising a first set of software instructions which, when executed by a peripheral computing platform on board an aircraft, implement the first processing phase of a method for automatic characterization of learning data as briefly described above and a second set of software instructions, which, when executed by a remote computing system, implement the second processing phase of a method for automatic characterization of learning data as briefly defined above.

[0037] According to another aspect, the invention relates to a method for using learning data acquired by an aircraft and characterized by an automatic characterization method as briefly described above, comprising a step of machine learning the parameters of a plurality of object detection and classification models, each model being trained on learning data as a function of a homogeneity criterion of at least one characterization information item associated with the data.

[0038] According to another aspect, the invention relates to a device for using learning data acquired by an aircraft as briefly described above.

[0039] The invention will appear more clearly on reading the description which follows, given solely by way of non-limiting example, and made with reference to the drawings in which:

[0040] [Fig-1] [Fig.l] is a synoptic of an aircraft and a characterization system automatic training data associated in an example implementation;

[0041] [Fig.2] [Fig.2] is a flowchart of the main steps of a method for automatically characterizing learning data according to one embodiment;

[0042] [Fig.3] [Fig.3] is a flowchart of the steps of a method for using models trained by machine learning on training data obtained by the characterization method.

[0043] [Fig.l] schematically shows a system for automatic characterization of learning data 2.

[0044] The system 2 comprises an aircraft 4, for example an aircraft without a pilot on board.

[0045] Alternatively, the aircraft 4 is another type of aircraft, for example a helicopter, or any other type of aircraft, with or without a pilot on board, and adapted for flying at low altitude.

[0046] The aircraft 4 comprises one or more digital image sensors 6, and various certified instruments 8, providing information on the spatial location, attitude, and speed of the aircraft 4, as well as an avionics calculation system 10, integrating calculations and functionalities certified according to an avionics certification standard, in accordance with the avionics safety standards imposed for the aircraft 4.

[0047] For example, the or each digital image sensor 6 is an optical camera, positioned so that the shooting field is oriented towards the outside of the aircraft. Preferably, the exact orientation of each digital image sensor 6, in particular the orientation for shooting, is controlled and known, for example in a spatial reference system associated with the aircraft.

[0048] The digital image sensors 6 form an optical acquisition system.

[0049] Preferably, this optical acquisition system is calibrated, by intrinsic and extrinsic calibration.

[0050] It should be noted that the term “digital images” designates a succession of digital images captured with a known acquisition frequency, forming a video stream.

[0051] According to the embodiments, successive digital images, recorded individually or in video form, are time-stamped, so an associated capture time is recorded. Such an acquisition time is time information (or time-stamping information), obtained for example by an internal clock of the sensors 6 or of the avionics computing system 10.

[0052] In addition, the aircraft 4 comprises an on-board peripheral computing platform 12.

[0053] Unlike the avionics computing system 10, the peripheral computing platform 12 is configured to implement complementary calculations compared to the calculations implemented by the avionics computing system 10, and includes connection capabilities with non-certified external systems.

[0054] The peripheral computing platform 12 comprises a first communication interface 14, configured to communicate with the avionics instruments 6, 8 and the avionics computing system 10 and more generally with all the on-board equipment certified according to avionics safety standards, and a second communication interface 16, allowing communication, by a wireless communication system, with external systems, not subject to avionics safety standards.

[0055] For example, the first communication interface 14 is configured to communicate with the certified on-board equipment by wired communication or by wireless communication, for example Wifi or Bluetooth.

[0056] The second communication interface 16 is for example configured to communicate with non-certified external systems, by a radio communication protocol, for example a standard 4G mobile telephony protocol. or 5G, the SATCOM satellite communication protocol, the Datalink AOC / ATC protocol (for “Aeronautical Operation Control” / “Air Traffic Control”).

[0057] The second communication interface 16 is for example configured to communicate with a remote computing system 20, comprising a plurality of interconnected computing servers adapted to communicate with each other via their own communication interfaces, and which provide computing services. Such a computing system 20 is for example a cloud computing infrastructure, more commonly called “cloud computing”.

[0058] The peripheral computing platform 12 further comprises a computing unit 22, comprising one or more processors (CPU or GPU), and at least one electronic memory unit 24, these elements being adapted to communicate via a communication bus.

[0059] In one embodiment, the peripheral computing platform 12 is a programmable electronic device.

[0060] In the example of [Fig.l] only one calculation unit 22 and one electronic memory unit 24 are shown.

[0061] The electronic memory unit 24 includes in particular memories of the RAM, ROM type, any type of non-volatile memory (for example FLASH, NVRAM).

[0062] The set of calculation unit(s) 22 and electronic memory unit(s) 24 forms the local calculation resources of the peripheral calculation platform 12.

[0063] In one embodiment, the peripheral computing platform 12 is modular, with multiple processing and memory units being able to be added, allowing for dynamic increase in available resources, as needed.

[0064] The calculation unit 22 is configured to execute a first data processing phase of the method for processing learning data acquired by the aircraft 4, these learning data being digital images acquired in flight by the aircraft 4 or being extracted from digital images acquired in flight by the aircraft 4.

[0065] In particular, the calculation unit 22 implements a module 26 for obtaining digital images acquired by the sensor(s) 6, of flight data obtained from the onboard avionics instruments 8; a coherence module 28, which performs a calculation of spatial and / or temporal location information associated with the acquired digital images, and a storage of the acquired digital images and the associated spatial and / or temporal location information, in a suitable storage structure 30 in the electronic memory 24.

[0066] The storage structure 30 is for example a file, a set of files, a database.

[0067] The modules 26 and 28 are adapted to cooperate, as described in more detail below, to implement the first processing phase of the characterization method. automatic training of data, as described in more detail below, according to various embodiments.

[0068] In one embodiment, the modules 26, 28 are produced in the form of software instructions forming a computer program, which, when executed by a computer, implements the first processing phase of the method for automatic characterization of learning data.

[0069] In a variant not shown, the modules 26, 28 are each produced in the form of programmable logic components, such as FPGAs (Field Programmable Gate Array), microprocessors, GPGPU components (General-purpose graphics processing), or even dedicated integrated circuits, such as ASICs (Application Specific Integrated Circuit).

[0070] The computer program comprising software instructions is further capable of being recorded on a medium, not shown, that is readable by a computer. The computer-readable medium is, for example, a medium capable of storing the electronic instructions and of being coupled to a bus of a computer system. By way of example, the readable medium is an optical disk, a magneto-optical disk, a ROM memory, a RAM memory, any type of non-volatile memory (for example FLASH, NVRAM), a magnetic card or an optical card.

[0071] Furthermore, the peripheral computing platform 12 is configured to transmit, via the second communication interface 16, via the communication link 25, the acquired digital images and associated spatial and / or temporal location information, to the remote computing system 20.

[0072] According to embodiments, the transmission takes place either as the digital images are acquired, when the aircraft is in flight, or after completion of a data collection flight, when the aircraft is on the ground, for example returning to a logistics base.

[0073] In one embodiment, the remote computing system 20 comprises or is connected to one or more interconnected programmable computing servers (not shown).

[0074] Thanks to the communication link 25 established via the second communication interface, the remote computing system 20 is in bidirectional communication with the peripheral computing platform 12.

[0075] This advantageously makes it possible to deport part of the calculations to be carried out to the remote calculation system 20, and thus to reduce the consumption of on-board computing and electrical resources. In other words, the peripheral calculation platform 12 cooperates with the remote calculation system 20 for the implementation of the calculations, the results of the calculations carried out by the external system being transmitted to the peripheral computing platform 12 via the bidirectional communication link 25.

[0076] The computing system 20 comprises a third communication interface 32, which is for example configured to communicate with the second communication interface 16, by a radio communication protocol, for example a standard 4G or 5G mobile telephony protocol, the SATCOM satellite communication protocol, the Datalink AOC / ATC protocol (for “Aeronautical Operation Control” / “Air Traffic Control”), and a fourth communication interface 34, adapted to communicate by wired or wireless link by a packet switching protocol, to make a connection to the Internet network.

[0077] Furthermore, the calculation system 20 comprises one or more calculation unit(s) 36 (processors) and one or more electronic memory unit(s) 38. In [Fig.l] a calculation unit 36 ​​and an electronic memory unit 38 are shown. These calculation and electronic memory units form calculation resources of the calculation system 20.

[0078] The calculation unit 36 ​​is configured to execute a second data processing phase of the method for processing learning data acquired by the aircraft 4.

[0079] In particular, the calculation unit 36 ​​implements:

[0080] - a module 40 for labeling objects contained in said digital images, making it possible to obtain a classification label for each object from among a plurality of predetermined classes;

[0081] -a module 42 for calculating characterization information associated with the objects as a function of the location, spatial and / or temporal information received and at least one piece of complementary information;

[0082] -a module 44 for storing each detected object in association with a label and at least one characterization information in at least one learning database 35.

[0083] Thus, the characterization information forms metadata, which provides data characterization information from the learning database 35, relating to the acquisition context for example.

[0084] The module 42 implements an analysis based on additional information extracted from recordings, for example from external databases 45, 46, for example from a geographic database, 45, i.e. a precise map of a geographic area, providing characteristics of the terrain of this area, and a database 46 of meteorological data.

[0085] Preferably, the module 42 implements geo-referencing of the detected objects, in a fixed terrestrial reference frame, which also makes it possible to correlate the positioning of detected objects with data from external databases 45, 46.

[0086] In one embodiment, the learning data are stored in several learning databases, 35A, 35B, according to the characterization information, so as to obtain specialized learning databases, in other words homogeneous according to certain homogeneity criteria of the characterization information associated with the learning data. As a non-limiting example, a homogeneity criterion is the homogeneity of the meteorological conditions for image acquisition, i.e. the separation between data acquired in sunny conditions and data acquired in rain / snow conditions, or the homogeneity of the types of backgrounds, i.e. background of fields or forests.The modules 40, 42, 44 are adapted to cooperate, as described in more detail below, to implement the second processing phase of the automatic method for characterizing learning data as described in more detail below, according to various embodiments.

[0087] In one embodiment, the modules 40, 42, 44 are produced in the form of software instructions forming a computer program, which, when executed by a computer, implements the second processing phase of the automatic method for characterizing learning data according to the invention.

[0088] In a variant not shown, the modules 40, 42, 44 are each produced in the form of programmable logic components, such as FPGAs (Field Programmable Gate Array), microprocessors, GPGPU components (General-purpose processing on graphics processing), or even dedicated integrated circuits, such as ASICs (Application Specific Integrated Circuit).

[0089] The computer program comprising software instructions is further capable of being recorded on a medium, not shown, readable by a computer. The computer-readable medium is, for example, a medium capable of storing the electronic instructions and of being coupled to a bus of a computer system. By way of example, the readable medium is an optical disk, a magneto-optical disk, a ROM memory, a RAM memory, any type of non-volatile memory (for example FLASH, NVRAM), a magnetic card.

[0090] Furthermore, the learning database(s) 35 are subsequently actually used for learning the parameters of an artificial intelligence model, for example a model for detecting and classifying objects present in digital images.

[0091] Advantageously, the associated characterization information is taken into account during this phase of learning the parameters of a detection model and classification by machine learning, and this allows specialization of models based on characterization information and ensures better performance in terms of precision and recall.

[0092] In a considered application case, such detection and classification models 48A, 48B, trained by machine learning on learning databases 35A, 35B, are retransmitted to the aircraft 4 for use in an operational phase, for example in a mission phase, for the detection of objects of a given class. The peripheral computing platform 12 then implements a module 50 for using the detection and classification models 48A, 48B by machine learning, the parameters of which have been learned on characterized learning databases.

[0093] An example of implementation of such a usage module 50 will be described below with reference to [Fig.3].

[0094] [Fig.2] is a synopsis of the main steps of a method for automatically characterizing learning data according to one embodiment.

[0095] As indicated above, the method comprises two processing phases, which are respectively a first phase 60 implemented by the on-board peripheral computing platform, in cooperation with the on-board sensors and instruments of the aircraft which carries out the acquisition of digital images, and a second phase 62 implemented by the remote computing system.

[0096] The first processing phase 60 is carried out when the aircraft is in flight.

[0097] The first processing phase comprises steps, carried out substantially in parallel acquisition of digital images 64, acquisition of flight data 66, and obtaining 68 of the positions of the digital image acquisition sensor(s).

[0098] Preferably, each step 64, 66 and 68 comprises an implementation of a timestamp.

[0099] Preferably, the digital images 64, the flight data 66 and the positions of the sensor(s) for acquiring the corresponding digital images are time-stamped relative to a common clock reference, for example an internal clock of the avionics computing system.

[0100] Preferably, the timestamping is precise, for example the timestamping is performed with an accuracy of the order of milliseconds.

[0101] The flight data include in particular the following time-stamped data: the attitudes of the aircraft (magnetic heading, attitude and inclination), for example provided in the fixed terrestrial reference frame (also called geo-referenced reference frame), the altitude of the aircraft, the speed of the aircraft, the speed component of the aircraft in the fixed terrestrial reference frame, geo-referenced spatial position information of the aircraft, for example provided by a GNSS (for “Global Navigation Satellite Systems”) receiver.

[0102] These data are provided as input to a step 70 of consistency between the acquired digital images and the flight data obtained, also using the positions of the digital image acquisition sensors.

[0103] During the consistency step 70, for each digital image of at least part of the acquired digital images, the method obtains a corresponding acquisition time instant (timestamp), and the position and attitude information of the aircraft at the same time instant, which makes it possible to calculate a correspondence between the specific reference frame of the digital image and the fixed terrestrial reference frame.

[0104] A spatial position of each digital image in the fixed terrestrial reference frame, or geo-referenced spatial position, is then obtained.

[0105] In addition, the attitude of the aircraft makes it possible to calculate a shooting angle, and the altitude of the aircraft makes it possible to determine a representation scale.

[0106] Thus, for each digital image of at least part of the acquired digital images, a set of location, spatial and / or temporal information is obtained.

[0107] Preferably, as explained above, the location information comprises spatial and temporal location information.

[0108] The location information is stored in association with the corresponding digital images, and is subsequently transmitted, during the transmission step 72, to the remote computing system.

[0109] The second processing phase 62 is carried out on the ground, by the remote computing system, for example by cloud computing processing.

[0110] The second processing phase comprises, after receiving the digital images and the associated spatial and / or temporal location information, several steps described below.

[0111] The second processing phase comprises a labeling (or annotation) step 80, i.e. associating a label (or tag) of a class from among a plurality of predetermined classes, with objects detected in each digital image.

[0112] The labeling step is implemented either by an automated process or by a semi-automated process with operator assistance.

[0113] The term objects is used here in the broad sense, an object being all or part of a digital image, belonging to a given class, depending on the intended application.

[0114] For example, in a road traffic monitoring system, the objects to be detected are to be classified between various types of vehicles (motorcycles, bicycles, automobiles, trucks), pedestrians, fixed obstacles.

[0115] The labeling task can then consist of segmenting each image to detect the objects of the classes sought, tracing a polygon, for example a rectangle, around each detected object, or in other words the polygon circumscribed to the detected object, and associating a label designating the associated class.

[0116] The method further comprises a step 82 of geo-referencing each detected object, i.e. calculating a spatial position in the fixed terrestrial reference frame.

[0117] Indeed, in the example explained, each polygon, for example each rectangle, has a position defined by its corners in the reference frame of the corresponding digital image, and the location information in the fixed terrestrial reference frame of the digital image is known, which makes it possible to calculate, by changing the reference frame, the geo-referenced spatial position of each object in relation to the corresponding temporal information.

[0118] For example, latitude and longitude coordinates are calculated for a chosen point, for example the geometric center of the rectangle circumscribing the detected object.

[0119] According to a variant, the polygon circumscribed to the detected object is gridded by regular tiles, and a point is associated with each tile of the grid, for example a central point or a corner of the tile, thus forming a set of points defining the object, each point having a geo-referenced spatial position.

[0120] The method then comprises a step 84 of calculating characterization information for each detected object, as a function of the spatial and / or temporal location information and at least one piece of complementary information.

[0121] In one embodiment, additional information is obtained from external databases (or other type of data recording), for example comprising meteorological data on the one hand, and cartographic data on the other hand.

[0122] For each object, its geo-referenced spatial position(s) and the associated temporal information make it possible to obtain, for example, reliably, meteorological characterization information (e.g. sunshine, rainfall, fog) or terrain characterization information (type of terrain).

[0123] Taking the road example again, it is then possible to determine the weather conditions and the type of road, crossroads or roundabout where the detected objects are located, without having to carry out a complex analysis of the digital images.

[0124] The second processing phase of the method then comprises a storage step 86, a learning database, objects detected in association with the associated label and at least one piece of characterization information.

[0125] The learning data thus labeled and characterized are then used, according to any known method, to automatically learn the parameters of a model for detecting and classifying objects into the set of predetermined classes. Any type of model trained by machine learning is applicable, for example a neural network of chosen architecture, for example a convolutional neural network or CNN (for "Convolutional Neural Networks"). The number of layers, number of neurons per layer and activation function implemented are chosen according to the application. The development of a neural network architecture for a given application is known per se and is not described here.

[0126] Alternatively, other detection and classification models trained by machine learning are usable.

[0127] Characterization of training data is for example used to separate the training data and generate better specialized machine learning trained detection and classification models based on the characterization information.

[0128] For example, in the case of obtaining training data from digital images acquired by an aircraft, the characterization information relating to the acquisition meteorological conditions is taken into account. This then makes it possible to obtain a first detection and classification model 48A trained by machine learning on training data acquired in sunny conditions, and a second detection and classification model 48B trained by machine learning on training data acquired in rainy conditions.

[0129] [Fig.3] is a synopsis of the main steps of a method of using machine learning-trained detection and classification models, specialized based on characterization information, in an operational phase.

[0130] For example, the method of use is implemented by an aircraft, in an operational phase of a mission, for example a rescue mission (in English “search and rescue”).

[0131] For example, the method of use is implemented by the on-board peripheral computing platform, after obtaining and storing the detection and classification models 48A and 48B trained by machine learning.

[0132] More generally, the method of use is implemented by a programmable computing device, forming a device for using learning data acquired by an aircraft and characterized by a method as described above.

[0133] The method comprises a step 90 of obtaining meteorological information in real time, thanks to the connectivity via the second on-board communication interface with remote systems, for example with a meteorological data server.

[0134] The method then comprises a determination 92 of the local meteorological conditions, i.e. as a function of the geo-localized spatial position of the aircraft, for example a level of rainfall, and a determination 94 of the detection and classification model to be used among the models 48A and 48B, as a function of the level of rainfall. The chosen detection and classification model is then applied (step 96) in real time for the detection and classification of objects flown over. The detection and classification performance is then better than in the case of the use of a single model, trained on uncharacterized training data, which risks being effective only on the majority training data.

[0135] Steps 90 to 96 are repeated in real time, which then makes it possible to automatically and dynamically adapt the choice of the detection and classification model to the actual meteorological conditions.

[0136] The use of characterized machine learning data has been described as an example in the particular case of taking into account additional meteorological information, but it is clear that other applications, for other types of additional information, are conceivable in a similar manner.

[0137] The invention has been described more particularly for detection and classification models trained by machine learning, but it applies to any type of model or method trained by machine learning, for example for prediction, segmentation, depth detection, etc.

Claims

1. Claims Method for automatically characterizing learning data acquired by an aircraft and used for machine learning of parameters of an artificial intelligence model, the method comprising an acquisition (64) of digital images by sensors (6) on board said aircraft, and being characterized in that it comprises a first processing phase (60) implemented by a peripheral computing platform (12), on board the aircraft, said peripheral computing platform (12) comprising a communication interface (16) configured to communicate with a remote computing system (20) comprising a plurality of interconnected computing servers, and a second processing phase (62) implemented by said remote computing system (20),in which the first processing phase (60) comprises a coherence (70) between at least part of the acquired digital images and flight data obtained from on-board avionics instruments, a storage of associated spatial and / or temporal location information, in association with said digital images, and a transmission (72) of said digital images and associated spatial and / or temporal location information to the remote computing system, and the second processing phase (62) comprises:, - a labeling (80) of objects contained in said digital images, making it possible to obtain a classification label for each object from among a plurality of predetermined classes; - a calculation of characterization information (84) associated with the objects as a function of the spatial and / or temporal location information, and of at least one piece of complementary information, the complementary information being obtained from a recording of meteorological data, and - a storage (86), in a learning database, of said detected objects in association with said label and at least one characterization information.

2. Method according to claim 1, comprising in the first processing phase (60) a storage of temporal information associated with each digital image, said temporal information being a time instant of acquisition of the digital image.

3. Method according to claim 2, in which said flight data are time-stamped, and the consistency (70) comprises, for each processed digital image, a calculation of a georeferenced spatial position of said digital image as a function of the flight data acquired at the time instant of acquisition of the digital image.

4. Method according to any one of claims 1 to 3, in which the second processing phase (62) comprises a calculation (82) of at least one geo-referenced position associated with each detected object.

5. Method according to claim 4, in which the step of labeling (80) objects comprises a detection, in each processed digital image, of one or more objects and of tracing a polygon circumscribed to each detected object, said at least one geo-referenced position being a position of a point of said polygon.

6. The method of claim 5, wherein said polygon is gridded into tiles, a point is associated with each tile and said object is associated with the geo-referenced positions of each of said points associated with the tiles.

7. Method according to any one of claims 1 to 6, wherein said at least one complementary information is obtained from at least one external data record.

8. System for automatic characterization of learning data acquired by an aircraft and used for machine learning of parameters of an artificial intelligence model, the system comprising an aircraft equipped with on-board sensors (6) configured to acquire digital images, the aircraft comprising a peripheral computing platform (12), the characterization system further comprising a remote computing system (20) comprising a plurality of interconnected computing servers, the peripheral computing platform (12) comprising a communication interface (16) configured to communicate with the remote computing system (20), the peripheral computing platform (12) comprising, for the implementation of a first processing phase, a module (28) for ensuring consistency between at least a portion of the acquired digital images and the flight data obtained

9.

10. from on-board avionics instruments, storage of associated spatial and / or temporal location information, in association with said digital images, and transmission of said digital images and associated spatial and / or temporal location information to the remote computing system, the remote computing system (20) comprising, for the implementation of a second processing phase: a module (40) for labeling objects contained in said digital images, making it possible to obtain a classification label for each object from among a plurality of predetermined classes; a module (42) for calculating characterization information associated with the objects as a function of the spatial and / or temporal location information, and at least one piece of complementary information, the complementary information being obtained from a recording of meteorological data, and a module (44) for storing, in a learning database (35), said detected objects in association with said label and at least one characterization information. Computer program, implemented in a system for automatic characterization of learning data according to claim 9, comprising a first set of software instructions which, when executed by a peripheral computing platform on board an aircraft, implement the first processing phase of a method for automatic characterization of learning data according to claims 1 to 7, and a second set of software instructions, which, when executed by a remote computing system, implement the second processing phase of a method for automatic characterization of learning data according to claims 1 to 7. Method for using learning data acquired by an aircraft and characterized by an automatic characterization method according to claims 1 to 7, comprising a step of machine learning the parameters of a plurality of object detection and classification models, each model being trained on learning data according to a homogeneity criterion of at least one characterization information associated with the data.

11. Device for using training data acquired by an aircraft configured to implement a method for using training data acquired by an aircraft according to claim 10.

Images