Method for controlling an autonomous vehicle with remote situation analysis

Abstract

Method of controlling a drone (1) by means of a remote station (11) for the accomplishment of a mission, the drone comprising a first electronic control unit (2) and a first data transceiver (4) connected to the first control unit and arranged to establish a communication channel (21, 22, 23) with a second transceiver (14) of the remote station, the remote station comprising a second control unit (12) connected to the second transceiver, characterized in that the method comprises the step of storing, before the start of the mission, in the first control unit and in the second control unit, a database (3, 13) containing predicted mission data relating to the mission to be accomplished, and the following steps carried out, during the mission: the first control unit collects current mission data and compares it to the predicted mission data,In the event of a discrepancy between the current mission data and the predicted mission data, the first command unit transmits data representing this discrepancy to the second command unit. The second command unit then transmits to the first command unit at least one action to be taken based on the data representing this discrepancy. (See diagram: Fig. 1)

Description

Title of the invention: Method for controlling an autonomous vehicle with remote situation analysis

[0001] The present invention relates to the field of remote-controlled, programmed and / or at least partially autonomous vehicles. These vehicles will be called drones in the remainder of the description.

[0002] BACKGROUND OF THE INVENTION

[0003] A drone is an aerial, spatial, nautical, terrestrial or amphibious vehicle which generally comprises a motorization to move the drone on a trajectory, at least one guidance member to modify the trajectory of the drone, on-board sensors and an electronic control unit executing a piloting program corresponding to a mission that the drone must accomplish. The mission always includes the movement of the drone from a starting point to an arrival point and for example the transport of passenger(s), the depositing of a load, surveillance by collecting information such as images or electromagnetic signals, the jamming of electromagnetic signals, the firing of a munition (the munition can be constituted by the drone itself)...

[0004] The electronic control unit is connected to the guidance member and to the on-board sensors, as well as possibly to the motorization, and the control program (or remote operation program) is arranged to control the steering member and the motorization according to the information provided by the on-board sensors which include positioning sensors (for example an inertial unit and / or a receiver of satellite positioning signals or GNSS) and environmental sensors allowing the electronic control unit to perceive its environment, such as optronic sensors, radar, lidar, sonar, etc.

[0005] It may happen during the movement of the drone that the electronic control unit detects an event not foreseen by the piloting program. This event is for example constituted by the occurrence of one or more obstacles on the trajectory of the drone, the absence of a target at the arrival point or on the contrary the presence of several targets at the arrival point, the detection of a potential target before reaching the arrival point, a fault, etc. Such an event can cause a disruption in the piloting of the drone, the electronic control unit being incapable of making a decision in the face of the unforeseen situation caused by the occurrence of this event (decisional impasse).

[0006] Ideally, the electronic control unit is arranged to contact a human operator capable of deciding or even carrying out actions to be carried out. taking into account this situation before allowing the electronic control unit to continue the mission. This assumes, however, that a communication channel can be established between the electronic control unit and the operator and that this communication channel offers sufficient throughput to allow the transmission to the operator of a data stream from the drone's sensors allowing him to understand the situation. This assumes that the drone has efficient means of communication and is located in an area favorable to communications (distance, relief, weather conditions, absence or presence of jamming, etc.).

[0007] SUBJECT OF THE INVENTION

[0008] The invention aims in particular to improve the piloting of a drone in such a situation. Summary of the invention

[0009] To this end, the invention provides a method for controlling a drone by means of a remote station for the accomplishment of a mission, the drone comprising a first electronic control unit and a first data transmitter / receiver connected to the first control unit and arranged to establish a communication channel with a second transmitter / receiver of the remote station, the remote station comprising a second control unit connected to the second transmitter / receiver. The method comprises the step of storing, before the start of the mission, in the first control unit and in the second control unit, a database containing predicted mission data relating to the mission to be accomplished, and the following steps carried out, during the mission: the first control unit collects current mission data and compares it to predicted mission data, in the event of a discrepancy between the current mission data and the predicted mission data, the first control unit transmits data representative of this discrepancy to the second control unit, the second control unit transmits to the first control unit at least one action to be carried out based on the data representative of this difference.

[0010] Thus, the first control unit can delegate the decision-making either directly to the second control unit or to an operator interacting with the second control unit. Transmitting only data representative of the difference between the predicted data and the current data makes it possible to limit the quantity of data transmitted while providing the second control unit or the operator with data enabling them to make a decision.

[0011] According to optional features, used individually or in whole or in part in combination : - the action to be carried out is determined by an operator to whom the second control unit has presented the data representative of the deviation; - the action to be carried out is determined automatically by the second control unit from the data representative of this difference; - the first control unit orders the data representative of the deviation in order of importance, determines a flow rate of the communication channel with the second control unit and transmits all or part of the data representative of the deviation according to their importance and the determined flow rate; - the representative data of the deviation are based on a difference between a predicted image and a current image; - the difference between the predicted image and the current image reveals the presence of an object and the data representative of the deviation include a type of object obtained by a classification algorithm, a position and dimensions of the object; - the predicted mission data includes data belonging to at least one of the following data groups: drone status data, terrain model data, obstacle map data, weather data, health data of at least one passenger of the drone, data related to the environment perceived by the drone; - the first control unit and / or the second control unit implements artificial intelligence; - the remote station is a fixed ground station; - the first control unit transmits data representative of the deviation to the second control unit only if the first control unit is in a decisional impasse given this deviation.

[0012] Other characteristics and advantages of the invention will emerge from reading the following description of a particular and non-limiting embodiment of the invention. Brief description of the drawings

[0013] Reference will be made to the accompanying drawings, among which:

[0014] [Fig-1] [Fig.l] is a schematic view of a drone control system usable for the implementation of the invention;

[0015] [Fig.2] [Fig.2] is a flowchart illustrating the implementation of the method according to the invention. DETAILED DESCRIPTION OF THE INVENTION

[0016] With reference to [Fig.l], the invention is described here in relation to a drone 1, here aerial, and a remote station 11, here fixed and on the ground G.

[0017] The drone 1 comprises a control unit 2 connected to a database 3, a transmitter / receiver 4, a group of sensors 5, a motorization 6 and a steering organ 7.

[0018] The control unit 2 is an electronic unit comprising, for example, one or more processors and at least one memory containing an operating system and one or more computer programs comprising instructions arranged to implement the method of the invention. The control unit is, for example, produced in the form of one or more integrated circuits, an FPGA or the like. The control unit 2 is powered by an on-board electrical network comprising at least one battery.

[0019] The database 3 is hosted in a memory embedded in the drone 1 and accessible by the control unit 2. The content of the database will be detailed in the rest of the description.

[0020] The transmitter / receiver 4 is a radio transmitter / receiver which is arranged to communicate via a communication channel with the remote station 11 and which preferably incorporates an electronic unit for encrypting and decrypting the transmitted data. The communication channel may be a direct communication channel 21, a satellite communication channel 22 passing through a communication satellite S or a relayed communication channel 23, here by an aircraft A (but it could be by a terrestrial relay or a maritime relay).

[0021] The group of sensors 5 here comprises: - status sensors of the various components / equipment of the drone 1; - an inertial navigation system comprising, in a manner known per se, accelerometers for detecting a specific force along the axes of a measurement frame and angular sensors of the orientation of the measurement frame relative to a reference frame; - a satellite positioning signal receiver (or GNSS receiver); - an optronic image sensor which operates in the visible and infrared ranges and which is oriented to capture images of the ground; - a radar to detect the possible presence of an obstacle in front of the drone 1.

[0022] The motorization 6 is arranged to propel the drone along an aerial trajectory and here comprises a propeller of the reactor, turbojet, ramjet, turbo-propeller type, etc.

[0023] The steering organ 7 here comprises orientable motorized fins whose orientation makes it possible to modify the trajectory of the drone 1.

[0024] The control unit 2 is programmed, in a manner known per se, to control the motorization 6 and the steering organ 7 so as to take the drone from a starting point to an arrival point by implementing a na algorithm hybridized navigation using signals from the inertial navigation unit and the GNSS receiver.

[0025] The control unit 2 is programmed, in a manner known per se, to accomplish a mission consisting for example of capturing images along the trajectory by means of the optronic image sensor and firing a munition at a target identified on one of the images.

[0026] The remote station 11 comprises a control unit 12 connected to a database 13, a transmitter / receiver 14, and a user interface 15.

[0027] The control unit 12 is an electronic unit comprising, for example, one or more processors and at least one memory containing an operating system and one or more computer programs comprising instructions arranged to implement the method of the invention. The control unit 12 is, for example, produced in the form of one or more integrated circuits, an FPGA or other.

[0028] The database 13 is hosted in a memory embedded in the remote station 11 and accessible by the control unit 12. The content of the database will be detailed in the remainder of the description.

[0029] The transmitter / receiver 14 is a radio transmitter / receiver which is arranged to communicate via a communication channel with the drone 11 and which preferably incorporates an electronic unit for encrypting and decrypting the transmitted data.

[0030] The user interface 15 here comprises at least one data input device such as a keyboard, a joystick or the like allowing an operator to enter data and / or instructions into the control unit 12 and at least one data output device such as a screen allowing the operator to view data presented by the control unit 12.

[0031] Both the control unit 2 and the control unit 12 here implement artificial intelligence to perform the previously mentioned functions.

[0032] The databases 3 and 13 are intended to contain predicted mission data comprising here: - drone state data, - data from a terrain model on the planned trajectory with terrain images originating for example from a prior reconnaissance, - meteorological data on the planned trajectory (temperature, atmospheric pressure, visibility).

[0033] These predicted mission data are determined during a preliminary mission preparation phase and can be determined based on past missions, simulations and other models, prior reconnaissance, etc. and are recorded in the databases 3 and 13 during an initial synchronization phase of the databases. data 3 and 13.

[0034] Furthermore, during this same preliminary phase, the arrival point, the planned trajectory or obligatory passage points, and the operations to be carried out during the mission (here firing a munition at a target located at the arrival point) are programmed in the control unit 2.

[0035] The mission begins when drone 1 sets off on the planned trajectory.

[0036] The piloting and navigation of the drone 1 to its arrival position is ensured by the control unit 2 which, throughout the trajectory, also collects current mission data including: - images of the ground directly above its trajectory; - status data of the various components / equipment of the drone 1; - weather data.

[0037] The current mission data are processed as and when by the control unit 2 which compares them to the predicted mission data from the database 3. In the absence of a difference between the predicted mission data and the current mission data, the control unit 2 does not execute any particular action.

[0038] Otherwise, in the event of a discrepancy between the current mission data and the predicted mission data: - the control unit 2 transmits data representative of this difference to the control unit 12, - the control unit 12 transmits to the control unit 2 at least one action to be carried out based on the data representative of the deviation.

[0039] In the example illustrated in [Fig.2], the terrain model of the predicted mission data includes an image showing no vehicle at position X.

[0040] Comparison of the image captured at position X by the optronic image sensor of the sensor group 5 and the image of the predicted mission data reveals a discrepancy, here the presence of an object. The control unit 2 therefore updates the database 3 by recording the current image there in place of the predicted image.

[0041] The control unit 2 then determines the flow rate of the communication channel between the transmitter / receiver 4 and the transmitter / receiver 14.

[0042] If this flow rate is sufficient, the control unit 2 requests the synchronization of the databases 3 and 13.

[0043] If this flow rate is insufficient, the control unit 2 is programmed to process the difference between the predicted image and the current image to create data representative of the deviation comprising a type of object obtained by a classification algorithm, a position and estimated dimensions of the object. In this case, the object is identified as being a vehicle and more particularly a combat tank.

[0044] The control unit 2 is programmed to order the representative data of the deviation (those mentioned previously and possibly others) in order of importance and transmits all or part of the data representative of the deviation according to their importance and the determined flow rate. This means that in the event that the connection between the transmitters / receivers 4, 14 is maintained long enough, all the data representative of the deviation will be transmitted to the control unit 12. On the contrary, if the connection is interrupted or the flow rate is too low to transmit all the data, only the priority data (the most important) will be transmitted.

[0045] Upon receipt of the data representative of the deviation, the control unit 12 is arranged to: - either decide for itself on an action to be carried out by the control unit 2, - either display on the user interface screen 15 the data represented deviation indicators, with or without processing of the control unit 12.

[0046] In the first case, the action to be carried out (by action, we mean here an individual action or a series of actions) is determined automatically by the control unit 12 from the data representative of this difference.

[0047] In the second case, the action to be performed is determined by an operator who communicates the action in question by means, for example, of the keyboard of the user interface 15 or by selecting the action in question from among possible actions presented by the control unit 12 in a contextual menu displayed on the screen of the user interface 15. The possible actions are, for example, determined by the artificial intelligence of the control unit 12 from the data representative of this deviation.

[0048] The control unit 12 determines the action to be carried out, for example by using artificial intelligence or by implementing predetermined logic.

[0049] Once the action to be performed has been determined, the control unit 12 develops corresponding instructions and transmits them to the control unit 2 via the transmitters / receivers 14, 4.

[0050] The control unit 2 then executes the action in question.

[0051] The action in question may be: - interrupt the mission and return to the starting position; - stay above the X position and monitor the tank's movements; - fire ammunition at the tank; - ignore the tank and continue with the initial mission;

[0052] Of course, the invention is not limited to the embodiment described but encompasses any variant falling within the scope of the invention as defined by the claims.

[0053] In particular, the drone and / or the remote station may have a structure different from that described.

[0054] The group of sensors 5 may include other sensors replacing or adding to those mentioned previously: - a lidar, - a sonar, - an optronic sensor in the infrared or visible range, - an electromagnetic signal sensor, - a threat detector and for example a missile or firing launch detector, - one or more health sensors of one or more passengers of drone 1 (measuring passenger temperature, capturing facial images, measuring blood oxygen levels, measuring heart rate or respiratory rate, etc.),

[0055] The mission data may include other data replacing or adding to those mentioned previously: - data from an obstacle map, - health data of at least one passenger of the drone, - data representative of a climatic situation, - data on the presence of friendly and / or enemy forces in a context military...

[0056] The discrepancy between the predicted mission data and the current mission data may also concern: - a drop in visibility; - a very low temperature causing frost on the optics or making the drone 1 heavier; - weather conditions (gust of wind, storm) or damage making it difficult to stabilize the drone; - a failure of an on-board sensor; - the unforeseen presence of humans around a target, the command unit 2 being unable to determine whether they are civilians or enemies; - a deterioration in the health of a passenger, the control unit 2 being unable to determine whether to return the drone to its starting point, divert it to make an emergency landing, or continue to the planned arrival point; - an evolution of the environment perceived by the drone (the data compared may for example concern the environment perceived by the drone and in particular the state of certain objects in a scene in order for example to monitor the destruction / construction of buildings);

[0057] It can be provided that the control unit 2 transmits the data representative of the deviation to the control unit 12 only if the control unit 2 is in a decisional impasse (it is blocked or in an emergency situation) taking into account this deviation.

[0058] It should be noted that the drone may have more information than the remote station. This is not a problem and the databases can be synchronized later.

[0059] It will also be noted that the remote station can send information to a drone, for example when the remote station is in contact with several drones and one of the drones has less information than the others.

[0060] It has been indicated that the control unit 12 displays on the screen of the user interface 15 the data representative of the deviation, with or without processing. This processing is for example the correction of a blur or a white balance in the case of images. Other processing operations are of course conceivable.

[0061] The drone can be any land, water, air, amphibious, space vehicle, etc., at least partially autonomous. This vehicle can be designed to transport one or more passengers, a payload (surveillance equipment, ammunition, etc.), merchandise, etc. The vehicle can be a missile or a prowling munition, etc.

[0062] The remote station can be a mobile station on board a vehicle such as a land vehicle, a boat, an airplane, etc. or a fixed ground station.

Claims

Claims

1. Method for controlling a drone (1) by means of a remote station (11) for the accomplishment of a mission, the drone comprising a first electronic control unit (2) and a first data transmitter / receiver (4) connected to the first control unit and arranged to establish a communication channel (21, 22, 23) with a second transmitter / receiver (14) of the remote station, the remote station comprising a second control unit (12) connected to the second transmitter / receiver, characterized in that the method comprises the step of storing, before the start of the mission, in the first control unit and in the second control unit, a database (3, 13) containing predicted mission data relating to the mission to be accomplished, and the following steps carried out, during the mission: - the first control unit collects current mission data and compares them with the predicted mission data,- in the event of a discrepancy between the current mission data and the predicted mission data, the first control unit transmits data representative of this discrepancy to the second control unit, - the second control unit transmits to the first control unit at least one action to be carried out based on the data representative of this discrepancy.,

2. Method according to claim 1, comprising the following step: the action to be carried out is determined by an operator to whom the second control unit (12) has presented the data representative of the deviation.

3. Method according to claim 1, in which the action to be carried out is determined automatically by the second control unit (12) from the data representative of this deviation.

4. Method according to any one of the preceding claims, in which the first control unit (2) orders the data representative of the deviation in order of importance, determines a flow rate of the communication channel (21, 22, 23) with the second control unit (12) and transmits all or part of the data representative of the deviation according to their importance and the determined flow rate.

5. A method according to any preceding claim, wherein the data representative of the deviation is based on a difference between a predicted image and a current image.

6. The method of claim 5, wherein the difference between the predicted image and the current image reveals the presence of an object and the data representative of the deviation comprises a type of object obtained by a classification algorithm, a position and dimensions of the object.

7. Method according to any one of the preceding claims, in which the predicted mission data comprises data belonging to at least one of the following data groups: - drone state data, - terrain model data, - obstacle map data, - meteorological data, - health data of at least one passenger of the drone, - data related to the environment perceived by the drone.

8. Method according to any one of the preceding claims, wherein the first control unit (2) and / or the second control unit (12) implement(s) artificial intelligence.

9. Method according to any one of the preceding claims, in which the remote station (11) is a fixed ground station (G).

10. Method according to any one of the preceding claims, in which the first control unit (2) transmits data representative of the deviation to the second control unit (12) only if the first control unit is in a decisional impasse taking into account this deviation.

Images