Autonomous vehicle with sensors

Abstract

The invention relates to an autonomous unmanned aerial vehicle (E) comprising motor-driven propulsion means (30, 31), a control unit (P) for controlling the propulsion means in order to cause the vehicle to follow a flight path provided by a control unit, at least one sensor (C1, C2, C3) capable of delivering data that require digital processing, wherein the machine is provided with an onboard digital processing circuit system (1300) and is connected to at least one remote processing circuit system (2300, 3300) via a wireless communication network (WCC). It comprises, according to the invention: - a distribution circuit (1200) capable of directing data from the or each sensor to the onboard processing circuit system or to the remote processing circuit system according to at least one criterion from among the type of processing to be performed, the processing capacity of the onboard processing circuit system and of the remote processing circuit system, the processing availability of the onboard processing circuit system and of the remote processing circuit system, the capacity of the wireless communication network, the availability of the wireless communication network, and a data processing speed setpoint; - a circuit (1400) for recombining data processed by the onboard processing circuit system and the remote processing circuit system; and - a control unit interface circuit (1700), this being configured to account for the recombined processed data at a given time and to generate instructions for adjusting the behaviour of the vehicle.

Description

Title: Autonomous vehicle with sensors

[0001] Field of invention

[0002] The present invention generally relates to autonomous vehicles, for example of the aerial drone type (or UAV for “Unmanned Aerial Vehicle” in English), equipped with sensors.

[0003] State of the art

[0004] Such devices are becoming increasingly sophisticated, and a significant development in their use in recent years has been in the area of ​​collecting data of extremely diverse natures.

[0005] Thus, such devices can be equipped with sensors for measuring environmental data (meteorological data such as wind, humidity, temperature, brightness, cloudiness, pollutants, radioactivity, radiofrequency signals, to name just a few examples), or image sensors (in the visible or non-visible domains).

[0006] Some sensor data does not require any special processing, and is simply formatted to be transmitted to the ground via an appropriate communication channel.

[0007] Other sensor data requires significant digital processing. In particular, when several types of data are measured, resynchronization, combination, interpolation, or even prediction processing, for example using deep learning, can require significant computing resources in terms of processing power and memory capacity.

[0008] Furthermore, it may be desirable to adjust the machine's route based on the measurements taken, and the heavier the treatments, the greater the electrical power consumed, and the more difficult it will be to obtain good machine responsiveness based on the results of the measurement treatments.

[0009] Summary of the invention

[0010] The present invention aims to overcome all or part of these limitations of the state of the art.

[0011] To this end, an autonomous piloted aerial vehicle is proposed, comprising motorized propulsion means, a control unit for the propulsion means to cause the vehicle to follow a flight trajectory provided by a piloting unit, the vehicle further comprising at least one sensor capable of delivering data requiring digital processing, the vehicle being provided with on-board digital processing circuitry, and being connected to at least one remote processing circuitry via a wireless communication network, the vehicle being characterized in that it comprises: - a distribution circuit capable of directing data from the or each sensor to the on-board processing circuitry or to the remote processing circuitry according to at least one criterion among a type of processing to be carried out, a processing capacity of the on-board processing circuitry and the remote processing circuitry, a processing availability of the on-board processing circuitry and the remote processing circuitry, a capacity of the wireless communication network, an availability of the wireless communication network, a data processing rate instruction, - a circuit for recombination of data processed by the on-board processing circuitry and the remote processing circuitry, and - an interface circuit with the control unit, the latter being configured to take into account said recombined processed data at a given time and generate instructions for adjusting the behavior of the machine.

[0012] Some advantageous but optional aspects of such a device include the following additional features, taken individually or in any combination that the person skilled in the art will understand to be technically compatible with each other:

[0013] * instructions for adjusting the behavior of the machine are included in a group comprising: - flight adjustment instructions, - instructions for adjusting the capture using the sensor(s), - instructions for adjusting the digital processing methods for data from the sensor or at least one sensor.

[0014] * said interface circuit is configured to generate a map of the recombined processed data, with a view to dynamically determining the trajectory of the machine.

[0015] * the adjustment instructions include flight adjustment instructions capable of modifying the limits of a sweep of an area with the machine, and / or the speed of movement of the machine, and / or the accuracy of the sweep of the area.

[0016] * the data provided by at least one sensor are data characterizing the atmosphere of the area.

[0017] * the processing to be carried out on the data from at least one sensor includes algorithmic processing.

[0018] * the algorithmic processing includes at least one of spectral decomposition processing, matrix calculation processing, learning-based processing, and graph traversal processing.

[0019] * at least part of the remote processing circuitry is accessible via a radio network.

[0020] * at least part of the remote processing circuitry is accessible via a wide area network.

[0021] * at least one criterion is determined using a processing resource management device.

[0022] * the processing resource management device is common to all the machines.

[0023] * the processing resource management device implements a management protocol using a signaling channel of the wireless communication network.

[0024] Brief description of the drawings

[0025] Non-limiting embodiments of a machine according to the present invention are now described in detail with reference to the accompanying drawings, in which:

[0026] Fig. 1 is a schematic perspective view of a machine according to the invention,

[0027] Fig. 2 is a block diagram of the digital architecture of a machine according to the invention and of digital resources available remotely,

[0028] Fig. 3 is a more detailed block diagram of a particular implementation of the digital architecture of Fig. 2, and

[0029] Fig. 4 is a detailed block diagram of a possible implementation of on-board and off-board processing resources.

[0030] Detailed Description of Preferred Embodiments

[0031] With reference to Fig. 1, an autonomous flying machine E or drone is shown, here of the six-rotor type, which comprises a main body 10 and a set of arms 20 supporting as many rotors 30 each equipped with a propeller 31. Intrinsically, such a machine comprises an inertial unit 40 capable of delivering kinematic data, and a positioning unit 50, for example to the GPS standard, capable of delivering universal positioning data. It is also conventionally equipped with one or more cameras 60, the images of which are sent back to the ground via an appropriate communication channel.

[0032] The device finally comprises one or more electronic cards 100, ensuring, in a manner known per se, the autopilot functions, wireless communication with a non-embedded digital environment and in particular a ground station, and various on-board calculation means with processors, memories, input / output interfaces, etc. The assembly is powered by one or more rechargeable batteries not shown.

[0033] The drone is equipped with a set of other sensors, here three sensors C1, C2, C3, which we will call here “business” sensors depending on the type of application envisaged.

[0034] These sensors can be based on extremely diverse technologies, including physical, optical, acoustic, chemical, and deliver data which, after possible formatting, are made up of digital data of varying volume, at rates that can be extremely variable.

[0035] With reference to Fig. 2, the sensors C1, C2, C3 deliver signals S1, S2, S3 which can be pre-processed at the level of dedicated circuits or units 1110, 1120, 1130 (or possibly a common circuit). Such pre-processing can consist of different operations chosen in particular from an analog / digital conversion, normalization, filtering, formatting, etc. depending on the type of sensor and the design of the processing circuits located downstream.

[0036] More specifically, this preprocessing may include, for example, operations such as formatting in the form of messages, each containing a data field containing the data to be processed from a sensor, sensor identifier information, timestamp information, scheduling data, derived for example from the timestamp data, and any other useful data or metadata, such as (i) data provided by other elements such as an inertial unit, other sensors conventionally fitted to the machine and not requiring processing, such as a temperature sensor or distance sensors, and / or (ii) data indicative of the digital processing capacity required for the data contained in the message.Each unit 1110, 1120, 1130 can also, depending on the type of data received from a sensor, or the type of sensor having generated the data, insert into the message data representative of processing instructions to be executed, for example in the form of the address of a sub-program to be executed at the level of a local processing circuit (typically in “edge computing” in English terminology) or of a remote processing circuit.

[0037] In cases where data from different sensors are supplied at different rates, interpolation techniques are used, for example, to resynchronize them.

[0038] Note that for some types of sensors, the pre-processing unit is integrated into the sensor.

[0039] The digital signals SPT1, SPT2, SPT3 output from the preprocessing circuits 1110, 1120, 1130 are received by a distribution circuit 1200.

[0040] This circuit 1200 is configured to route the signals SPT1, SPT2, SPT3 to a digital signal processing resource chosen from a set of digital processing resources. The digital processing operations can be of very diverse natures, and more or less heavy depending on the nature of the processing and the volume of data contained in the signals SPT1, SPT2, SPT3. These can be, for example, spectral decompositions, vector or matrix calculations, statistical calculations, etc., in the time and / or spatial domains. These processes can also implement deep learning mechanisms.

[0041] A first resource includes digital processing means 1300 on board the machine (processor(s) and memory). These means can be shared with other on-board processing, or dedicated means.

[0042] Where the processing to be carried out lends itself to it (for example matrix calculations, FFT-type spectral decompositions or others, etc.), these on-board processing means advantageously include parallel processors according to “multi-core” architectures.

[0043] A second digital processing resource 2300 is located in a remote digital processing environment 2000, with which the machine communicates via a wireless transmission channel WCC according to an appropriate technology. A preferred transmission channel relies on cellular technologies of the “5G” type.

[0044] These remote digital processing means are implemented here in a radio network environment (RAN for “Radio Access Network” in English), one of the important aspects being the flow rate and speed of communication via the WCC channel and the processing speed at the level of the remote processing means.

[0045] Alternatively or in addition, as we will see later, these remote digital processing means are implemented in a wide area network environment, for example of the "Cloud" type, with potentially unlimited available resources and processing speeds dependent on the throughput and latency of transmission on the network.

[0046] These different processing means are capable of performing different types of processing on the pre-processed signals SPT1, ST2, ST3, distributed by the circuit 1200, as indicated above.

[0047] The signals processed by the remote processing means are returned to the circuits of the machine via the WCC channel, and all of the processed signals ST 1 , ST2, ST3, whether by the on-board intelligence 1300 or by the remote intelligence 2300, are combined at the level of a reordering or recombination circuit 1400 located in the machine. This circuit receives the processed signals in the form of messages, and the role of the circuit 1400 is to reorganize the processed data received, in particular in their time sequence, to make them usable at the level of a control unit of the machine E, generally designated by P in Fig. 2, for purposes which will be detailed later.

[0048] We will now describe with reference to Fig. 3 an example of a detailed architecture for the implementation of the principles illustrated in Fig. 2.

[0049] In the on-board circuitry 100, comprising the on-board digital processing means 1000, a single sensor C1 is shown, here a gas sensor, which generates raw data S1 which are collected by the pre-processing unit 1110 which may consist of a pipelined processor.

[0050] This unit 1110 includes: - a module 1111 for collecting raw data from the sensor (interface), - a module 1112 configured to add to the raw data different types of metadata, for example data included in a group comprising the GPS coordinates of the machine, its altitude, its speed, wind data (obtained by a specific sensor or from another data source), by carrying out the necessary interpolations in cases where this information is delivered at different rates, and by generating messages of appropriate format and - a module 1113 for queuing messages by adding a sequence number to them.

[0054] These messages are applied to the input of the distribution module 1200 responsible for distributing the messages to different digital processing units configured to apply the necessary processing to the sensor data, including the on-board processing unit 1300.

[0055] In the present implementation, there is a remote processing unit 2300 deployed at the level of an architecture 2000 accessible by a RAN radio network to which the device is connected, with for example a cellular network architecture of the “5G” type, and another remote processing unit 3300 deployed at the level of a wide area network environment 3000 of the “Cloud” type, which may include reduced latency processing means of the “Edge Network” type.

[0056] The distribution unit 1200 performs the distribution according to at least one criterion.

[0057] A preferred criterion is the need to ensure real-time or near-real-time processing of the data produced by the C1 sensor, particularly in the case where this processing is likely to impact the trajectory of the machine, as will be detailed below.

[0058] In this case, the distribution of the data messages to be processed is carried out according to a latency criterion and information on the availability and / or capacity of the processing units 1300, 2300, 3300.

[0059] If necessary, in the event that compliance with this criterion is impossible given the load on the processing units, the distribution unit can simply reject certain messages, which will not be processed but which can nevertheless be stored for later processing in deferred time.

[0060] According to another possible criterion, the distribution unit can send the data messages to be processed to the local processing unit 1300, monitor the return of this unit in terms of latency and possibly messages unprocessed, and in the event that the latency becomes greater than a certain threshold, distribute the data messages to be processed between the local unit 1300 and the remote processing units 2300, 3300. The distribution keys can be dynamically adjusted according to the latency measured at the level of the return messages from the processing units.

[0061] The above criterion can be subject to many variations. For example, we can favor the 2300 processing unit located at the radio network level, for the reliability of its connection with the machine, or even the processing unit located at the "Cloud" level, for its potentially unlimited processing capacity, with however greater exposure to latency phenomena.

[0062] In another approach, the circuits embedded in the machine can determine a processing capacity per unit of time required, based on a number of criteria.

[0063] For example, with a device having a given flight autonomy, having to cover a certain measurement path (for example by scanning parallel lines) and with an imposed measurement step (for example a measurement every X cm), the on-board circuitry (or a remote circuitry) can calculate the speed that the device must have to cover the entire path taking into account its available autonomy (with a safety margin), which in turn determines the frequency at which the sensor will send data to be processed (the higher the speed must be, the higher the frequency will be and the greater the volume of data to be processed per unit of time will be).

[0064] Based on this calculation, the distribution unit will decide to allocate the necessary resources between the local processing unit 1300 and the remote processing units 2300, 3300 so that this volume of data can be processed.

[0065] The recording unit (log function) of the sensor data is designated by the reference 1500. It comprises a unit 1510 for storing the raw data (here the signals S1 collected at the level of the unit 1111) and a unit 1520 for storing the processed data.

[0066] In delayed time, this data can be transmitted in batches to the processing means 2300 and / or 3300 for additional processing.

[0067] In one embodiment, the metadata insertion module 1112 may also incorporate preprocessing functions that may include: - as indicated above, an interpolation, which is implemented in the case of several sensors operating at different frequencies, and possibly non-synchronously (in particular the case of sensors having their own clock); - statistical calculations (calculations of moving averages, variances or standard deviations, etc.).

[0069] The processed data messages sent by the processing means 1300, 2300, 3300 are recovered at the reordering / recombining unit 1400 (illustrated in FIG. 3 as being located in the same block as the unit 1200) and reordered by the unit 1400 thanks in particular to the order numbers appearing in the messages. Preferably, the unit 1400 is associated with a latency management unit 1250, for example by rejecting packets received beyond a certain time limit (TTL) and / or packets received out of order (“jitter”).

[0070] The unit 1400 is connected via the unit 1250 on the one hand to the memory 1520 of processed data of the recording unit 1500, and on the other hand to a message agent 1600 (“message broker” in English) responsible for routing the messages as will now be described.

[0071] The agent 1600 redirects the messages (or certain messages, depending on their types) to a data analysis module 1700 configured to extract from the succession of messages processed by the units 1300, 2300, 3300 a certain number of metrics which can be extremely varied: average noise level, spectral decomposition with a view to detecting gas concentration peaks, sound signatures (case of acoustic sensors), spectral signatures in visible or invisible radiation, etc. The module 1700 can also be configured to carry out different kinds of post- processing on the recombined processed signals, these processings being able to include filtering, noise elimination, threshold detections, the creation of measurement maps, etc. Alternatively or in addition, such processings can also be carried out at the level of the recombination unit 1400.

[0072] The agent 1600 also redirects messages received from the unit 1250 and likely to impact the navigation of the craft, as well as analysis messages from the module 1700, to a navigation module 1800.

[0073] This module 1800 comprises a set of software 1100 (software stack) ensuring the navigation of the craft by generating piloting commands based either on flight instructions received from a ground station, or on the geolocation of the craft E and a trajectory to be followed, stored or dynamically generated based on a certain number of criteria including the flight environment. The module 1800 applies the flight instructions CV to a flight controller 1900, which returns telemetry values ​​TM to the module 1800 in a manner known per se.

[0074] Here, this module 1800 is also configured to adapt the behavior of the craft E according to the measurements made by the sensor C1 after processing by the means 1300, 2300, 3300 and, if necessary, analysis by the module 1700. In a typical implementation, this adaptation comprises an automatic adjustment of the trajectory of the craft (with respect to a set trajectory in the case of an autonomous flight). For example, when the sensor C1 and the associated processing operations are designed to carry out an environmental measurement (physical, optical, chemical, radioactivity parameters, etc.), the trajectory of the craft can be modified (turn back, travel another imposed line, etc.) when the processed sensor data show that the measured information becomes lower than a certain threshold and over a certain extent and / or for a certain duration.

[0075] It is thus possible to optimize the route of an area (for example a vertical plane starting from the ground, perpendicular to the direction of the wind) by limiting overlaps and avoiding traveling through regions of the area where the measured parameter is close to zero or insignificant, thus limiting the path of the machine to what is necessary. In this case, the machine typically includes a wind sensor to determine by calculation, using the wind vector, the orientation of the vertical (or possibly inclined) plane containing the path of the machine.

[0076] It is also possible, thanks to this function, to optimize the speed of the machine and / or the sampling and processing frequency according to certain characteristics of the processed measurements, in particular in the case where the evolutions of the captured signals become rapid and finer spatial sampling is desired.

[0077] In some cases, during a multi-sensor implementation, measurements made at one of the sensors may lead to temporarily ceasing to make other measurements using another sensor, either because these other measurements become unnecessary or less relevant, or to reserve a greater part of the processing capacity for the data of a particular sensor, possibly by increasing its sampling and processing time frequency and / or by decreasing its measurement step. This can be done either within the unit 1800, or in a specific sensor management unit, not illustrated.

[0078] Furthermore and in a related manner, the sensor data can be applied in whole or in part to an artificial intelligence, hosted in one or more of the processing units 1300, 2300, 3300, intended to adjust the operating conditions of the machine and the operating conditions of the sensors (modification of the trajectory and speed of the machine, starting or stopping a certain capture, modification of the capture frequency, choice of the processing unit among the processing units 1300, 2300, 3300 for the data from each of the sensors, with, where appropriate, specific arbitration rules).

[0079] Still referring to Fig. 3, the reference 2000 designates the off-board digital processing environment, connected to the machine via a radio network architecture and capable of contributing to the pre-processed signal processing available at the distribution module 1200. The digital processing resources themselves are designated by the reference 2300 and will be described in more detail below.

[0080] Finally, the reference 3000 designates the off-board digital processing environment, connected to the machine via a “Cloud” type architecture and also capable of contributing to the processing of pre-processed signals available at the level of the distribution module 1200. The digital processing resources themselves are designated by the reference 3300 and will be described in more detail below.

[0081] Please note that the data recovered by the “Cloud” 3000 environment can be processed in different ways, including: - in real time during flights; - in deferred time; - via an application programming interface (API) to make the data available for other uses.

[0082] In particular, in the case where the preprocessed data from a sensor could not be processed in real time, the environment 3000 can execute the processing algorithms in deferred time after downloading the raw data.

[0083] Still with reference to Fig. 3, the “Cloud” environment 3000 comprises a gateway 3100 with the engine E, which is connected to a set of data management services 3200. The data is routed to a sensor data processing service 3400 which makes the connection with the processing means 3300. Conventionally, the services 3200 are also connected to databases 3500, application programming interface (API) gateways 3600 for cooperation with third-party applications 3650, and client gateways 3700 for cooperation with client stations 3750.

[0084] With reference to Fig. 4, an architecture has been illustrated which may be common to the processing means 1300, 2300, 3300.

[0085] It comprises a secure proxy 310 and a processing architecture 320 comprising a manager 321 of the computing load cooperating with a planning unit (“scheduler” in English) 322, and a set of processing cores 323a-323h. As illustrated in the left part of Fig. 4, each processing core comprises an input / output interface 324 and a processor 325 executing a certain algorithm, receiving the input data DE resulting from the pre-processing and restoring the output data DS which will return to the reordering / recombination unit 1400.

[0086] In the case where the invention is intended to be implemented with a plurality of machines using the local / remote processing architecture as described above, advantageously, in relation to the allocation of resources mentioned above, a common device is provided, according to a centralized or distributed architecture, making it possible to manage the processing resources available locally at the level of the machines and remotely. This device may have the following tasks in particular: - planning of machine missions based on available processing resources, availability data can be determined by a schedule, response time measurements, etc. - the declaration of availability or need for processing resources, with for example an indication of the processing capacity available or requested and the period of availability and the period requested, this data being typically able to feed the calendar data used for the planning described above, - the implementation of a time resource discovery mechanism using an appropriate protocol, each accessible processing device being able to receive a request to this effect, - the implementation of time-to-live (TTL) management of packets circulating in the implementation of the architecture of the invention.

[0087] Advantageously, this common resource management device is implemented by relying on the signaling channels of the networks implemented.

[0088] Of course, the present invention may be the subject of numerous variations and improvements.

[0089] In particular, certain types of sensor data to be processed (image sensors or others) can advantageously be subject to compression and / or encryption before being sent to a remote processing unit 2300 or 3300. Furthermore, the data from the sensors can be signed in order to certify their authenticity.

[0090] The invention finds application in many fields and in particular: - the determination of the properties of an atmosphere (concentration of pollutants, particles, chemical agents, etc.), - the detection of specific events (fires, crowd movements, road traffic, snow, etc.) by image analysis, - verification of an event or measurement determined by external means (sensors on a land-based site), alert and evacuation system in the event of danger to the population, etc.

[0091] It applies in particular to the processing of data delivered by a single sensor, in particular a sensor delivering complex signals at high frequency, whether physical, electrical, optical, chemical, etc., or by a plurality of sensors of similar or different types.

Claims

Claims 1. Autonomous piloted aerial vehicle (E), comprising motorized propulsion means, (30, 31) a control unit for the propulsion means to cause the vehicle to follow a flight path provided by a piloting unit (P, 1800, 1900), the vehicle further comprising at least one sensor (C1, C2, C3) capable of delivering data requiring digital processing, the vehicle being provided with on-board digital processing circuitry (1300), and being connected to at least one remote processing circuitry (2300, 3300) via a wireless communication network (WCC), the vehicle being characterized in that it comprises: - a distribution circuit (1200) capable of directing data from the or each sensor to the on-board processing circuitry (1300) or to the remote processing circuitry (2300, 3300) as a function of at least one criterion among a type of processing to be carried out, a processing capacity of the on-board processing circuitry and of the remote processing circuitry, a processing availability of the on-board processing circuitry and of the remote processing circuitry, a capacity of the wireless communication network, an availability of the wireless communication network, a data processing rate instruction, - a circuit (1400) for recombination of data processed by the on-board processing circuitry and the remote processing circuitry, and - a circuit (1700) for interface with the control unit (P, 1800, 1900), the latter being configured to take into account said recombined processed data at a given time and generate instructions for adjusting the behavior of the machine.

2. Machine according to claim 1, characterized in that the instructions for adjusting the behavior of the machine are included in a group comprising: - flight adjustment instructions, - instructions for adjusting the capture using the sensor(s), - instructions for adjusting the digital processing methods for data from the sensor or at least one sensor.

3. Machine according to claim 1 or 2, characterized in that said interface circuit (1700) is configured to generate a map of the recombined processed data, with a view to dynamically determining the trajectory of the machine.

4. Machine according to claim 2, characterized in that the adjustment instructions comprise flight adjustment instructions capable of modifying the limits of a sweep of an area with the machine, and / or the speed of movement of the machine, and / or the precision of the path of the swept area.

5. Machine according to one of claims 1 to 4, characterized in that the data provided by at least one sensor are data characterizing the atmosphere of the area.

6. Machine according to one of claims 1 to 5, characterized in that the processing to be carried out on the data from at least one sensor comprises algorithmic processing.

7. Machine according to claim 6, characterized in that the algorithmic processing comprises at least one of a spectral decomposition processing, a matrix calculation processing, a learning-based processing, a graph traversal processing.

8. Machine according to one of claims 1 to 7, characterized in that at least one part (2300) of the remote processing circuitry is accessible via a radio network.

9. Machine according to one of claims 1 to 8, characterized in that at least one part (3300) of the remote processing circuitry is accessible via a wide area network.

10. Machine according to one of claims 1 to 9, characterized in that at least one criterion is determined using a device (1200) for managing processing resources.

11. Set of machines according to claim 10, characterized in that the device (1200) for managing processing resources is common to the set of machines.

12. Assembly according to claim 11, characterized in that the processing resource management device (1200) implements a management protocol using a signaling channel of the wireless communication network.

Images