Method and device for assigning a mission to each drone of a plurality of drones

Abstract

Method and device for assigning a mission to each drone of a plurality of drones The present invention relates to a method (50) for assigning a mission to a plurality of drones, the mission of which is to remotely explore points of an area, comprising the following steps: - obtaining (52) a terrain model of said area and the positions of said points forming a subset; - partitioning (54) said subset into groups; - assigning (56) to each drone a list of group(s); - for each drone, determining (58) a route order of said groups; iteratively until complete exploration of said subset: - tracking (60) the position and route of each drone until detection of sufficient exploration of a group and / or change of an exploration priority, and / or addition or deletion of a drone; - updating (62) the list, or - reiteration from the partitioning of the unexplored part. Figure for abstract: Figure 3

Description

Title of the invention: Method and device for assigning a mission to each drone of a plurality of drones

[0001] The present invention relates to a method for assigning a mission to each drone of a plurality of drones, each drone being equipped with at least one image sensor whose orientation and / or zoom can be controlled, the mission of the plurality of drones being to remotely explore a plurality of predetermined points of interest within a predetermined geographical area, the method being implemented by an electronic device for assigning a mission to each drone of a plurality of drones.

[0002] The invention also relates to a computer program comprising software instructions which, when implemented by a programmable electronic device, implement such a method.

[0003] The present invention also relates to an electronic device for assigning a mission to each drone of a plurality of drones configured to implement such a method.

[0004] Drone observation of large predetermined geographical areas often involves a very large number of shots for a single drone. This is often time-consuming. By using several drones in parallel, the time to observe each area among the predetermined areas is reduced. However, often the coordination of a plurality of drones (i.e. drone swarms) is complex and difficult to implement by an operator.

[0005] The field of the invention therefore relates to the efficient coordination of several drones or swarms of drones, in particular UAV (Unmanned Aerial Vehicle) aerial drones, potentially heterogeneous, and their detection means comprising in particular a camera with control of the orientation and / or zoom of the image sensor, with a view to successively and in parallel observing a set of points (i.e. elements) of interest, such as a road, a forest edge, the edges of buildings, etc. in a predetermined geographical area and / or to fill a lack of observation carried out by other detection means external to the swarm (satellite image type, high-altitude reconnaissance aircraft, ground vehicles).

[0006] More precisely, we seek to observe the points of interest efficiently, namely as quickly as possible, with potentially several shots, in particular partially orthogonal, and with adaptation capabilities to support a change in the area to be inspected during the mission, taking into account zone(s) over which overflight is prohibited (from the English No Fly Zones) during the mission, the taking into account the priority of observation of certain points (i.e. elements) of interest or type of points of interest, potentially during the mission, or even taking into account a change in the number of drones available during the mission, in particular, when implementing a rotation within the swarm of drones for the management of their energy autonomy.

[0007] By effective observation, we also mean an observation in an exhaustive manner (i.e. of all points of interest), or alternatively, in a partially exhaustive manner, with 90% of the surface to be observed as a threshold for example, if this brings a notable gain on the total duration of observation of the zone.

[0008] It should be noted that the swarm of drones is generally, for the needs of the mission, coupled with a video analysis system (from the English Video Analytics) capable of implementing automatic detection of objects of interest in the image(s) captured by the image sensor(s) of each drone, the system being configured to raise alerts when it detects objects of interest, typically human beings in a search and rescue context (from the English Search & Rescue).

[0009] However, when the size of the mission terrain is large, typically greater than 1km2, certain drone swarm control optimization solutions do not scale, i.e. do not have the capacity for quasi-instantaneous calculations for a large number of points of interest to be observed.

[0010] Typically, when the individual control of drones, relating to their geographical position and the orientation of the onboard image sensor, is carried out by a neural network, in the context of reinforcement learning, as implemented in particular according to patent application FR 2213738 in the name of the Applicant, the size of the state space of the neural network is limited. In other words, the neural network does not perceive the entire environment of the mission area, the size of which is large, typically greater than 1 km2, but only a sub-part.

[0011] In this context of individual control of drones carried out by means of a neural network, within the framework of reinforcement learning, for the observation of an area whose surface is greater than that suitable for being handled by the neural network whose state space size is limited, it is therefore conventionally proposed to proceed by moving a predetermined number of sliding windows whose size is suitable for being handled by the neural network whose state space size is limited (i.e. the size of each sliding window is small enough to be completely included in the state space of a neural network).

[0012] The problem that we seek to solve is therefore to optimize, for each drone, the movement of these sliding windows, by finding the most efficient sequence of movements fast in travel time, allowing remote observation of a potentially very large number of points of interest.

[0013] The most similar classical problem to the one above that we are trying to solve is the one concerning the optimization of a travel distance known as the traveling salesman problem (TSP). This problem determines a minimum distance between a plurality of cities, while visiting each city once. When several agents travel between cities, the problem is called the multiple traveling salesman problem (MTSP).

[0014] However, the MTSP formulation of the problem does not take into account the possibility of each drone to observe points of interest remotely, so that it is not necessary to be present above a point of interest to observe it. On the contrary, it may be faster to position oneself between two points and change the orientation of the camera's image sensor without moving, rather than moving successively above the two points of interest.

[0015] Furthermore, it is important to note that the set of points of interest to be observed, and therefore their positions, is known in advance on the one hand, and that it is a subset of the overall mission area. Consequently, all conventional technical solutions that deal with the discovery of the environment during the mission, such as the solution relating to simultaneous localization and mapping (SLAM) or any other environment discovery solution, are irrelevant, as are the techniques of systematic scanning of an entire surface of the "robot lawnmower" or "vacuum cleaner" type.

[0016] In other words, current solutions are unsuitable for taking both the remote vision capacity of drones and the need for scaling corresponding to a capacity for quasi-instantaneous calculations for a large number of points of interest to be observed, the order of magnitude of the number of points of interest to be observed being, for example, for a surface area of ​​10km2, discretized into a window (i.e. cell) of 10m2, for which we have 30% of the surface area to explore (i.e. only 30% of the points in the window are points of interest to be observed) of 300,000 points to explore.

[0017] An aim of the present invention is thus to optimize the allocation of observation window(s) to each drone of a plurality of drones controlled individually by means of a neural network, within the framework of reinforcement learning, and this by considering that only a sub-part known a priori, namely the points of interest, of the mission zone is to be observed, by taking into account the remote vision capacity of each drone, and by being capable of scaling up, that is to say of taking into account almost instantaneously the number of points of interest to be observed.

[0018] To this end, the subject of the invention is a method for assigning a mission to each drone of a plurality of drones individually controlled by means of a neural network, each drone being equipped with at least one image sensor whose orientation and / or zoom can be controlled, the mission of the plurality of drones being to remotely explore a plurality of predetermined points of interest within a predetermined geographical area, the method being implemented by an electronic device for assigning a mission to each drone of a plurality of drones, the method comprising at least the following steps:

[0019] - obtaining a digital model of the terrain associated with said geographical area predetermined and the position of each of said points of interest of said plurality within said digital model of said predetermined geographic area, said points of interest forming a subset of said predetermined geographic area, said digital model being an image of said predetermined geographic area, said image comprising a predetermined number of pixels distributed within said image according to a predetermined two-dimensional grid;

[0020] - partitioning said subset into exploration groups grouping together neighboring points of interest being spaced from each other by a distance less than a predetermined distance threshold, each group being of substantially equal size;

[0021] - depending on the current position of each of said drones, allocation to each of said drones from a list of exploration group(s) to be browsed, said list comprising at least one of said exploration groups;

[0022] - for each of said drones, determination of an order of travel of said groups exploration groups allocated when said list includes at least two of said exploration groups of said partitioned subset;

[0023] and the following steps implemented iteratively during said mission until said subset is completely explored:

[0024] - monitoring, in parallel, of the position and path of each of said drones, until detection of at least one of the following events:

[0025] - according to a predetermined criterion, of a sufficiency of exploration of at least one group current exploration allocated;

[0026] - changing an exploration priority of at least one of said points of interest;

[0027] - adding or removing a drone from said plurality;

[0028] - at least after each detection, updating of the list of exploration group(s) remaining distance to be covered allocated to each drone by removing from said list any sufficiently explored exploration group, or

[0029] - when adding or deleting a drone or changing exploration priority, reiteration of all steps starting from the partitioning step applied to the unexplored part of said subset.

[0030] The method proposed according to the present invention thus makes it possible to accomplish the observation mission as quickly as possible, because it focuses on the only sub-part known a priori of the environment to be observed, namely the points of interest, and this by taking into account the remote vision capacity of each drone, with a scaling up to a very high number of points of interest to be observed, and by being applicable to the multiple traveling salesman problem.

[0031] Indeed, according to the present invention, as a prerequisite, a digital terrain model pre-exists, and the positions of the elements (i.e. points) of interest are therefore known in advance. These elements of interest form a subset to be observed of the surface i.e. (mission zone). This subset of elements of interest corresponds within the image of the mission zone to a subset of pixels.

[0032] After loading the digital model of the terrain that the drones must explore, the method according to the present invention then proposes to divide the subset of points of interest / pixels to be visited into exploration groups (i.e. clusters in English) of similar size which group together points of interest close to each other, and to allocate them, at least partially, in the form of a list, to each drone of the swarm of drones assigned to the mission, by further determining for each drone the order of travel of said allocated exploration groups when said list comprises at least two of said exploration groups of said partitioned subset.

[0033] The mission is then launched and a second phase of the process begins. This involves a quasi-permanent updating of the lists of clusters to be explored assigned to each drone. This updating, based on the decision to consider an exploration group (i.e. an exploration window) as explored and move on to the next group, is capable of taking into account a multitude of factors in order to balance the workload of each drone while guaranteeing a certain level of efficiency of the solution provided.

[0034] According to other advantageous aspects of the invention, the method comprises one or more of the following characteristics, taken in isolation or in all technically possible combinations:

[0035] - said partitioning of said subset into exploration groups is a k-means partitioning, the optimal number of exploration groups being determined by:

[0036] - testing in parallel several values ​​of the number of exploration groups,

[0037] - by summing, for each value of the number of exploration groups, the surface of the convex hull associated with each exploration group, the optimal number corresponding to the value providing the minimum total surface area;

[0038] - said partitioning of said subset into exploration groups is obtained by tiling by linear programming:

[0039] - said partitioning of said subset into exploration groups is obtained by applying a predetermined tiling heuristic, each exploration group corresponding to a tile of said tiling, the application of said tiling heuristic comprising the following first steps:

[0040] - placing a first tile of predetermined shape in an arbitrary corner of said image, and

[0041] - obtaining the final position of said first tile within the image by:

[0042] - shift according to a first of the directions of the two-dimensional grid as that no pixel corresponding to one of said points of interest is outside said first tile, then by

[0043] - shift along the other direction of the two-dimensional grid as long as no pixel corresponding to one of said points of interest is not outside said first tile;

[0044] the points of interest covered by the first tile no longer being taken into account for determining the position of the following tiles;

[0045] and, until each of said points of interest of said plurality is covered, repeating the following steps:

[0046] - placement of another tile of said predetermined shape:

[0047] - directly following the tile whose final position was previously obtained according to said first direction, or,

[0048] - according to said other direction:

[0049] - at the beginning of the line following that of the tile whose final position has been previously obtained, if said other direction is horizontal, or

[0050] - from the column following that of the tile whose final position has been previously obtained, if said other direction is horizontal, and

[0051] - obtaining the final position of said other tile within the image by:

[0052] - shifting along said first direction as long as no pixel corresponding to a of said points of interest is not outside said other tile, then by

[0053] - shifting along said other direction as long as no pixel corresponding to one of said points of interest are outside of said other tile;

[0054] the points of interest covered by said other tile no longer being taken into account for determining the position of the following tiles;

[0055] - said method further comprises obtaining an exploration priority of at least one of the said points of interest, and its consideration during the said allocation,

[0056] said allocation comprising all of the following sub-steps:

[0057] - calculation of the surface of the convex hull associated with each group exploration;

[0058] - first allocation, to said plurality of drones, of all the exploration groups including at least one point of interest to be explored as a priority and / or from the nearest exploration group;

[0059] - determination of the drone having the least surface area to explore corresponding to the drone whose list of exploration group(s) to be traveled presents the minimum sum of the surfaces of the convex envelopes of said exploration groups which were allocated to it during said first allocation

[0060] - for each unassigned exploration group of said partitioned subset, determination of a score including:

[0061] - determining a first distance corresponding to the minimum distance said exploration group not assigned to said exploration groups assigned to said drone having the least surface area to explore;

[0062] - determining a second distance corresponding to the minimum distance said exploration group not assigned to said exploration groups assigned to the other drones of said plurality distinct from said drone having the least surface area to explore;

[0063] - determining the score corresponding to the difference between said first and second distances;

[0064] - until the sum of the areas of the convex hulls of said groups exploration groups allocated to each drone is substantially equal or until all said exploration groups resulting from the partitioning are allocated, second allocation of exploration groups not allocated during said first allocation, the drone having the least surface area to explore being allocated the unallocated exploration group with the lowest score.

[0065] - the determination of said order of travel includes the optimization of the distance of a route starting at the current position of said drone considered and passing through all of said exploration groups assigned to it by solving, by means of a 3-opt algorithm, a problem corresponding substantially to the traveling salesman problem or the multiple traveling salesman problem,

[0066] and considering a predetermined limit number of assigned exploration groups, said traversal order being re-optimized via said 3-opt algorithm as soon as one of its assigned exploration groups is sufficiently explored, by adding the assigned exploration group closest to the last of the assigned exploration groups belonging to the predetermined limit number of exploration groups assigned during the previous iteration of the 3-opt algorithm.

[0067] - said update further comprises a reassignment of said exploration groups remaining to be covered when the updated list of at least one of said drones is empty, said reallocation corresponding at least to the allocation, to said drone whose list is empty, of at least one exploration group remaining to be covered assigned to the drone having the most remaining surface area to explore, said at least one reassigned exploration group remaining to be covered being the furthest or the last to be visited according to the order of travel of said drone having the most remaining surface area to explore.

[0068] - said allocation comprises all of the following sub-steps:

[0069] - calculation of the convex hull of all the exploration groups of said sub- partitioned set;

[0070] - for each of said drones:

[0071] - determination of the vertex of the convex envelope closest to said drone considered ;

[0072] - determination of the exploration groups of said partitioned subset the most close to the said summit,

[0073] - among said exploration groups of said partitioned subset the most close to said summit, determination of said exploration group closest to said drone considered;

[0074] - allocation of said exploration group closest to said drone considered.

[0075] - said predetermined criterion depends, for each exploration group, at least on the following elements:

[0076] - the density of pixels to be explored corresponding to the proportion of pixels to be explored, each pixel to be explored corresponding to a point of interest to be explored, among the set of pixels constituting the current exploration group;

[0077] - the exploration rate corresponding to the proportion of pixels already explored among the set of pixels to explore in the current exploration group;

[0078] - the derivative of the exploration rate;

[0079] a plurality of distinct pixel density ranges being defined, each pixel density range being respectively associated with a pair of thresholds comprising a scanning rate threshold and a derivative threshold of said scanning rate;

[0080] for the density range of the current exploration group, the exploration sufficiency being obtained when the exploration rate is greater than said exploration rate threshold of said pair associated with said density range and when the exploration rate is less than the derivative threshold of said exploration rate.

[0081] The invention also relates to a computer program comprising software instructions which, when executed by a computer, implement a method of assigning a mission to each drone of a plurality of drones individually controlled by means of a neural network, as defined above.

[0082] The invention also relates to an electronic device for assigning a mission to each drone of a plurality of drones controlled individually by means of a neural network, each drone being provided with at least one image sensor of which the orientation and / or zoom is controllable, the mission of the plurality of drones being to remotely explore a plurality of predetermined points of interest within a predetermined geographical area, the electronic device for assigning a mission to each drone of a plurality of drones comprising at least:

[0083] - an obtaining module configured to obtain a digital model of the terrain associated with said predetermined geographic area and the position of each of said points of interest of said plurality within said digital model of said predetermined geographic area, said points of interest forming a subset of said predetermined geographic area, said digital model being an image of said predetermined geographic area, said image comprising a predetermined number of pixels distributed within said image according to a predetermined two-dimensional grid;

[0084] - a partitioning module configured to partition said subset into exploration groups grouping together neighboring points of interest spaced from each other by a distance less than a predetermined distance threshold, each group being of substantially equal size;

[0085] - an allocation module configured to allocate to each of said drones a list of exploration group(s) to be browsed according to the current position of each of said drones, said list comprising at least one of said exploration groups of said partitioned subset;

[0086] - a determination module configured, for each of said drones, to determine an order of traversal of said assigned exploration groups when said list includes at least two of said exploration groups;

[0087] and the following modules, implemented iteratively during said mission until said subset is completely explored:

[0088] - a tracking module configured to track in parallel the position and the path of each of said drones, until detection of at least one of the following events:

[0089] - according to a predetermined criterion, of sufficient exploration of at least one group current exploration allocated;

[0090] - changing an exploration priority of at least one of said points of interest;

[0091] - adding or removing a drone from said plurality;

[0092] - an update module configured to update, at least after each detection, the list of exploration group(s) remaining to be explored assigned to each drone by removing from said list any sufficiently explored exploration group.

[0093] Subsequently, such a device is also called a “high-level controller” hereinafter.

[0094] The invention will appear more clearly on reading the description which follows, given solely by way of non-limiting example, and made with reference to the drawings in which:

[0095] [Fig-1] [Fig.l] is a schematic representation of an electronic device assigning a mission to each drone of a plurality of individually controlled drones using a neural network;

[0096] [Fig.2] [Fig.2] illustrates an example of the various associated input(s) and output(s) electronic mission allocation device of [Fig.l];

[0097] [Fig.3] [Fig.3] is a flowchart of a mission allocation process to each drone of a plurality of drones individually controlled by means of a neural network, the method being implemented by the device of [Fig.l];

[0098] [Fig.4] [Fig.4] illustrates an example of tiling heuristics implemented for the partitioning said subset of points of interest of said geographic area;

[0099] [Fig.5] [Fig.5] illustrates a first variant of implementation of said step of attribution;

[0100] [Fig.6] [Fig.6] illustrates a first variant of implementation of said step of attribution;

[0101] [Fig.7] [Fig.7] illustrates the pair of thresholds used to detect sufficiency exploration of a current exploration group.

[0102] In the remainder of the description, the expression “substantially equal to” is understood as a relationship of equality to plus or minus 10%, that is to say with a variation of at most 10%, more preferably as a relationship of equality to plus or minus 5%, that is to say with a variation of at most 5%.

[0103] The electronic device 10 for assigning a mission to each drone of a plurality of drones individually controlled by means of a neural network according to the invention, also called a high-level controller, is illustrated in [Fig.l]. As can be seen in this [Fig.l], the electronic device for assigning a mission to each drone of a plurality of drones comprises an obtaining module 12 configured to obtain a digital model of the terrain associated with said predetermined geographical area and the position of each of said points of interest of said plurality within said digital model of said predetermined geographical area, said points of interest forming a subset of said predetermined geographical area, said digital model being an image of said predetermined geographical area, said image comprising a predetermined number of pixels distributed within said image according to a predetermined two-dimensional grid.

[0104] As shown in [Fig.l], the electronic device 10 for assigning a mission to each drone of a plurality of drones further comprises a partitioning module 14 configured to partition said subset into groups exploration grouping neighboring points of interest spaced from each other by a distance less than a predetermined distance threshold, each group being of approximately equal size.

[0105] In addition, the electronic device 10 for assigning a mission to each drone of a plurality of drones comprises an assignment module 16 configured to allocate to each of said drones a list of exploration group(s) to be browsed as a function of the current position of each of said drones, said list comprising at least one of said exploration groups of said partitioned subset.

[0106] Furthermore, the electronic device 10 for assigning a mission to each drone of a plurality of drones comprises a determination module 18 configured, for each of said drones, to determine a route order of said assigned exploration groups when said list comprises at least two of said exploration groups of said subset.

[0107] The electronic device 10 for assigning a mission to each drone of a plurality of drones also comprises the modules 20 and 22, implemented iteratively during said mission until said subset is completely explored, the module 20 corresponding to a tracking module configured to track in parallel the position and the path of each of said drones, until detection of at least one of the following events: according to a predetermined criterion, sufficient exploration of at least one currently assigned exploration group; change of an exploration priority of at least one of said points of interest; addition or deletion of a drone to said plurality; the module 22 corresponding to an update module configured to update, at least after each detection, the list of exploration group(s) remaining to be explored assigned to each drone by deleting from said list any sufficiently explored exploration group

[0108] In the example of [Fig.l], the electronic device for assigning a mission to each drone of a plurality of drones comprises an information processing unit 24 formed for example of a memory 26 and a processor 28 associated with the memory 26.

[0109] In the example of [Fig.l], the obtaining module 12, the partitioning module 14, the allocation module 16, the determination module 18, the tracking module 20 and the update module 22 are each produced in the form of software, or a software brick, executable by the processor. The memory of the electronic device for assigning a mission to each drone of a plurality of drones is then able to store obtaining software, partitioning software, allocation software, determination software, tracking software and update software. The processor is then able to execute each of the software among the obtaining software, partitioning software, allocation software, determination software, tracking software and updating software.

[0110] In a variant not shown, the obtaining module 12, the partitioning module 14, the allocation module 16, the determination module 18, the tracking module 20 and the updating module 22 are each produced in the form of a programmable logic component, such as an FPGA (Field Programmable Gate Array), or an integrated circuit, such as an ASIC (Application Specific Integrated Circuit).

[0111] When the electronic device 10 for assigning a mission to each drone of a plurality of drones is produced in the form of one or more software programs, that is to say in the form of a computer program, also called a computer program product, it is furthermore capable of being recorded on a medium, not shown, readable by a computer. The computer-readable medium is for example a medium capable of storing electronic instructions and of being coupled to a bus of a computer system. By way of example, the readable medium is an optical disk, a magneto-optical disk, a ROM memory, a RAM memory, any type of non-volatile memory (for example FLASH or NVRAM) or a magnetic card. A computer program comprising software instructions is then stored on the readable medium.

[0112] [Fig.2] illustrates an example 30 of the different input(s) and output(s) associated with said electronic mission allocation device of [Fig.l].

[0113] More precisely, according to this example 30, the electronic device 10 for assigning a mission to each drone of a plurality of drones according to the present invention is firstly configured to obtain, via the obtaining module 12 of [Fig.l], the digital model 32 of the terrain associated with said predetermined geographical mission zone.

[0114] Then, via its partitioning module 14, the electronic device 10 for assigning a mission to each drone of a plurality of drones according to the present invention is configured to partition, according to the arrow 34, only said subset into exploration groups illustrated by different shades of gray on the view 36. These exploration groups, also called clusters, group together neighboring points of interest by being spaced from each other by a distance less than a predetermined distance threshold, each group being of substantially equal size, in particular in terms of exploration duration, of which the surface of the convex envelope is for example representative.

[0115] These clusters are transmitted according to arrow 38 as input to the rest 40 of the modules of said device 10, namely the allocation module 16, the determination module 18, the tracking module 20 and the update module 22.

[0116] In parallel, the current position of each of said drones 42 is transmitted according to the arrow 44 also as input to the rest 40 of the modules of said device 10, namely the allocation module 16, the determination module 18, the tracking module 20 and the update module 22.

[0117] As indicated previously, from the clusters of the view 36, and from the current position of each of said drones 42 transmitted according to the arrow 44, the allocation module 16 of this remainder 40 of the modules is configured to allocate to each of said drones 42 a list 46 of exploration group(s) to be browsed, said list 46 comprising at least one of said exploration groups (i.e. cluster).

[0118] And according to the arrows 48 and 49, the electronic mission allocation device 10 is configured to continuously update, by means of its tracking module 20 configured to track in parallel the current position and the path of each of said drones 44 transmitted via said arrow 44 to said remainder of module 40, and by means of its update module 22 the list of exploration group(s) remaining to be covered allocated to each drone by deleting from said list any sufficiently explored exploration group.

[0119] According to one embodiment, each drone is capable of carrying its own electronic mission allocation device 10 (i.e. high-level controller). In this case, an additional synchronization mechanism is necessary. All the drones must then exchange, typically every thirty seconds, their positions and which exploration groups (i.e. clusters) they have assigned to each other. In the case where two separate drones allocate clusters in common, the one that chose first, which can be verified via a timestamp, keeps the exploration groups (i.e. clusters) while the other restarts the allocation process (i.e. allocation) this time with the information that the exploration groups mentioned are already allocated to another drone.

[0120] According to another embodiment, at least one electronic mission allocation device 10 (i.e. high-level controller) is embedded in a ground control station, which can itself be embedded in a vehicle).

[0121] A location means of the GNSS (Global Navigation Satellite Systems) satellite positioning type is also suitable for being embedded in each drone 42 to enable their location. Alternatively, the location means of the drone is suitable for corresponding to a coupling of the “inertial unit and video analysis” type of landmarks (Vision Based Navigation). According to another alternative, each drone is suitable for being located by means of a radar associated with the ground control station.

[0122] A means of automatic analysis of sensor data, for example by machine learning for image analysis, allowing objects to be classified by nature is embedded in drones. Typically, an optical (or infrared) camera is embedded and equipped with processing allowing automatic detection of vehicles, or the presence of humans.

[0123] Alternatively, the image processing is transferred to the ground control station and the image stream is sent via a means of communication, such as pTMA, 5G, Wifi, Satcom, etc. ... allowing information to be sent back to the control station when an object of interest is detected. Typically, adding a predetermined icon and its type in a graphical user interface (GUI) of the tactical situation, at the correct position, when a drone detects a vehicle of a predetermined type.

[0124] Furthermore, the list of pixels / cells observed so far is sent back to the control station at regular intervals, typically every second, or every ten seconds, or even every minute. This information feedback is shared within the swarm of drones if several drones are used. With the list of current positions of the drones, this view of what has been observed is a sufficient means of synchronization to allow the high-level partitioning implemented by the device according to the present invention to distribute the drones appropriately over the surface to be inspected corresponding to the mission. In other words, the information synchronization can be imperfect (updated every thirty seconds for example), without compromising the operation of the solution.

[0125] If one of the drones is deleted or added to the swarm, it is sufficient for the high-level partitioning module 14 to be executed again on the subset of points of interest not yet explored for the swarm to adapt to this new configuration.

[0126] A minimal and classic obstacle detection and anti-collision system not shown can be embedded within each drone. Note that the avoidance of trees or buildings is done quite spontaneously due to the flight altitude of the drones imposed by the use case associated with its mission 30m or 50m, 100m 200m or even 500m etc.

[0127] Furthermore, in a manner not shown, a conventional ground altitude control system (in particular so as to remain above ground level AGL) is suitable for being carried within each drone, allowing a “terrain following” type flight.

[0128] A method 50 for assigning a mission to each drone of a plurality of drones controlled individually by means of a neural network, implemented by the electronic device 10, will now be explained with reference to [Fig. 3] presenting a flowchart of the steps of this method 50 and to FIGS. 4 to 7 illustrating different variants of implementation thereof.

[0129] More precisely, as indicated previously, each drone is equipped with at least one image sensor whose orientation and / or zoom can be controlled, the mission of the plurality of drones being to explore (i.e. search, observe, search) remotely a plurality of predetermined points of interest within a predetermined geographical area.

[0130] The method 50 is implemented by the electronic allocation device described previously in relation to Figures 1 and 2 and comprises a first step 52 of obtaining a digital model of the terrain associated with said predetermined geographical area and the position of each of said points of interest of said plurality within said digital model of said predetermined geographical area, said points of interest forming a subset of said predetermined geographical area, said digital model being an image of said predetermined geographical area, said image comprising a predetermined number of pixels distributed within said image according to a predetermined two-dimensional grid.

[0131] In other words, according to this obtaining step 52, the pre-existing knowledge of the digital model of the terrain and of the position of the points of interest of the mission within the acquired model is acquired by the electronic mission allocation device according to the present invention.

[0132] The method 50 according to the present invention then comprises a step 54 of partitioning said subset into exploration groups grouping together neighboring points of interest being spaced from each other by a distance less than a predetermined distance threshold, each group being of substantially equal size, in particular in terms of exploration duration, the surface of the convex envelope of which is for example representative

[0133] Then, depending on the current position of each of said drones, in general, the method 50 according to the present invention comprises a step 56 of assigning to each of said drones a list of exploration group(s) to be browsed, said list comprising at least one of said exploration groups.

[0134] Then, the method 50 comprises, for each of said drones, a step 58 of determining a route order of said allocated exploration groups when said list comprises at least two of said exploration groups of said partitioned subset.

[0135] Then, the method 50 according to the present invention comprises a step 60 of tracking, in parallel, the position and the path of each of said drones, until detection of at least one of the following events:

[0136] - according to a predetermined criterion, of sufficient exploration of at least one group current exploration allocated;

[0137] - changing an exploration priority of at least one of said points of interest;

[0138] - adding or removing a drone from said plurality.

[0139] Finally, the method 50 according to the present invention comprises a step 62, implemented at least after each detection, of updating M_A_J the list of exploration group(s) remaining to be explored assigned to each drone by deleting from said list any sufficiently explored exploration group, or during an addition or deletion of a drone or a change of exploration priority by repeating (in a manner not shown) all of the steps from the partitioning step 54 applied to the unexplored part of said subset.

[0140] According to arrow 64, the tracking 60 and updating 62 steps are implemented iteratively during said mission until said subset is completely explored.

[0141] Each of these steps 52, 54, 56, 58, 60, 62 is described in more detail below.

[0142] In particular, during step 52 of obtaining the digital terrain model, all of the points of interest to be visited are provided in the form of a 2D two-dimensional grid. This grid contains full pixels (or of a predetermined color such as black) when they correspond to terrain elements to be observed (i.e. the points of interest of the current mission), or empty pixels in the opposite case (or of a predetermined color distinct from that associated with the pixel corresponding to terrain elements to be observed). In other words, the digital model is binary, the binarity resulting from the fact that a point of the geographical area of ​​the terrain of the mission is to be observed or not. As an optional addition, information relating to the obsolescence and / or redundancy of the observation of the point of interest is associated with each corresponding pixel.In other words, obsolescence means that the observation of the point of interest is too old compared to a predetermined duration, and redundancy means the need to repeat the observation, for example from different observation angles according to a predetermined angle difference between each observation angle.

[0143] As shown in dotted lines according to [Fig.3], said obtaining step 52 further comprises, optionally, the implementation of a sub-step 66 of obtaining terrain relief information O_IT suitable for being taken into account during the implementation of said partitioning to delimit said exploration groups formed subsequently during the partitioning step 54.

[0144] Indeed, as it stands, the exploration groups (i.e. the clusters) resulting from the implementation of partitioning step 54 are only a technical means and do not correspond to any operational notion of the mission. That said, according to option 66, these clusters could be calculated to be bounded, for example, by topology information and / or information relating to the relief of the terrain, ridge lines, among others, so as to then obtain, due to their consideration during the step 54 partitioning, clusters which correspond to the geotactic notion of “terrain compartment”.

[0145] As represented in dotted lines according to [Fig.3], said obtaining step 52 further comprises, optionally, the implementation of a sub-step 68 of obtaining an exploration priority of at least one of said points of interest, and its taking into account during said allocation 56.

[0146] Thus, when a user of the device 10 according to the present invention enters areas to be explored as a priority during the mission, it is easy to assign these priorities to the exploration groups (i.e. clusters) which correspond to these priority areas. Indeed, as indicated below, the solution for assigning clusters and that for traversing the clusters are easily modified to process the high priority exploration groups (i.e. clusters) first.

[0147] With regard to the next partitioning step 54, the solution adopted is generally based on spatial partitioning.

[0148] To put it simply, a "naive" implementation would divide the mission area into as many sectors as there are drones. A systematic traversal / sweeping algorithm, for example of the lawnmower type, could then produce the succession of windows that would allow everything to be covered successively.

[0149] If the state space of the neural network is a square of n pixels on a side, typically with n of the order of one hundred, and the terrain associated with the mission corresponds to an image area of ​​the order of a thousand pixels or cells on a side, with typically a pixel (or cell) being 10m, i.e. a terrain of 10km on a side, it is conventional to decompose this terrain into a regular 2D grid of n pixels per window / cell (the last cell row and / or the last column may be only partially filled with pixels).

[0150] Conventionally, if we have drones, it is known to divide the grid into m sets containing approximately the same number of cells (to the nearest integer rounding). A systematic scanning strategy can then be used: when all the pixels (or a threshold of the 90% type) of a cell have been observed by a drone, it is moved to the neighboring connected cell (i.e. window), hence the name sliding window, the size of each window being compatible with the state space of the neural network used. Note that conventionally it is also known to use such sliding windows but which overlap as long as the succession of windows, overlapping or not, covers the entire surface of the mission.

[0151] However, this classic approach to partitioning is not optimal because it is unsuitable for easily managing a notion of zone priority and / or because the windows / cells do not necessarily contain the same number of pixels corresponding to the points of interest to be observed specifically according to the mission. In other words, the durations necessary for observing the windows can be very variable, which potentially leads to a sub-optimal solution where drones finish their mission (no more window to observe) while others are far from the end, inefficiency(ies) that the present invention proposes to avoid by means of three distinct partitioning options 70, 72 and 74, more elaborate than the aforementioned classic spatial partitioning.

[0152] According to a first option 70, said partitioning of said subset into exploration groups is a K_M partitioning into k-means (i.e. K nearest neighbors) (from the English K-Means), the optimal number of exploration groups being determined by:

[0153] - testing in parallel several values ​​of the number of exploration groups,

[0154] - by summing, for each value of the number of exploration groups, the surface of the convex hull associated with each exploration group, the optimal number corresponding to the value providing the minimum total surface area.

[0155] More precisely, this type of partitioning (i.e. clustering) into k-means (i.e. K nearest neighbors) (K-means), is applied to the points of interest, obtained during the obtaining step 52, in order to extract exploration groups (i.e. clusters of points of interest). These clusters have the property of only grouping together points of interest that are relatively close to each other when they come from the same cluster.

[0156] Conventionally, the algorithm associated with partitioning (i.e. clustering) in k-means (i.e. K nearest neighbors) does not allow the optimal number of clusters (i.e. exploration groups) to be extracted to be automatically determined.

[0157] The preferability of a value of the number of clusters used is determined according to the present option of the present invention by the total surface area covered by the convex hulls of the clusters and / or the shape of these convex hulls. This total surface area of ​​the convex hulls and / or their shape is / are in fact a good indicator of the exploration time of the cluster(s). The sum of the areas of the hulls of all the clusters decreases as the number of clusters increases. Also, several values ​​of this total number k of clusters are tested in parallel, and when the decrease in the total surface area of ​​the convex hulls of the clusters begins to slow down, the value of the number k of clusters retained is the one which provided the minimum total area.

[0158] Note that in this approach according to option 70, certain clusters of the partitioning (i.e. clustering) in k-means (i.e. K nearest neighbors) (K-Means) may be larger in surface area than the square “cell” which is conventionally used as input to the neural network which acts as a low-level controller as notably implemented according to patent application FR 2213738 to name of the Applicant. A naive intra-cluster route is then classically implemented, of the lawnmower type for example.

[0159] According to another option 72 of the partitioning 54, said partitioning 54 of said subset into exploration groups is obtained by tiling by linear programming T-PL.

[0160] As an alternative to partitioning (i.e. clustering) in k-means (i.e. K nearest neighbors) (K-Means), it can be noted that the partitioning problem that one seeks to solve is ultimately a minimum coverage problem, namely finding a minimal cardinality set of windows (for example rectangular or square in shape to form the “cells” given as input to the neural network which acts as a low-level controller as notably implemented according to patent application FR 2213738 in the name of the Applicant) such that all the points to be observed (i.e. the points of interest) are at least in one window.

[0161] Linear programming tiling T-PL makes it possible to address such a minimum coverage problem. However, for certain scenarios, this linear programming tiling approach T-PL is not able to scale to the number of points of interest to be observed (i.e. does not have the capacity for quasi-instantaneous calculations for a large number of points of interest to be observed) the calculation times of linear programming tiling can be several minutes, which is incompatible with the need to be able to restart the calculations during the mission, following a change, for example, in the input hypotheses.

[0162] Also, according to another option 74 of the partitioning 54, said partitioning 54 of said subset into exploration groups is obtained by applying, according to a sub-step 74, a predetermined tiling heuristic T_H, each exploration group corresponding to a tile of said tiling.

[0163] More specifically, as described below in relation to [Fig.4], the application 74 of said tiling heuristic comprises the following first steps:

[0164] - placing a first tile of predetermined shape in an arbitrary corner of said image, and

[0165] - obtaining the final position of said first tile within the image by:

[0166] - shift according to a first of the directions of the two-dimensional grid as that no pixel corresponding to one of said points of interest is outside said first tile, then by

[0167] - shift along the other direction of the two-dimensional grid as long as no pixel corresponding to one of said points of interest is not outside said first tile;

[0168] the points of interest covered by the first tile no longer being taken into account for determining the position of the following tiles

[0169] and, until each of said points of interest of said plurality is covered, repeating the following steps:

[0170] - placement of another tile of said predetermined shape:

[0171] - directly following the tile whose final position was previously obtained according to said first direction, or,

[0172] - according to said other direction:

[0173] - at the beginning of the line following that of the tile whose final position has been previously obtained, if said other direction is horizontal, or

[0174] - from the column following that of the tile whose final position has been previously obtained, if said other direction is horizontal, and

[0175] - obtaining the final position of said other tile within the image by:

[0176] - shifting along said first direction as long as no pixel corresponding to a of said points of interest is not outside said other tile, then by

[0177] - shifting along said other direction as long as no pixel corresponding to one of said points of interest are outside of said other tile;

[0178] the points of interest covered by said other tile no longer being taken into account for determining the position of the following tiles.

[0179] As regards the next allocation step 56, as for the previous partitioning step 54, several options are possible.

[0180] According to a first option, as described below in relation to [Fig.5], as indicated previously when optionally the step 52 of obtaining also comprises the optional obtaining 68 of an exploration priority of at least one of said points of interest, and its taking into account during said allocation,

[0181] said allocation 56 then comprises the set 76 of the following sub-steps:

[0182] - calculation of the surface of the convex envelope associated with each group exploration;

[0183] - first allocation, to said plurality of drones, of all the exploration groups including at least one point of interest to be explored as a priority and / or from the nearest exploration group;

[0184] - determination of the drone having the shortest cumulative exploration time corresponding to the drone whose list of exploration group(s) to be traveled presents the minimum sum of the surfaces of the convex envelopes of said exploration groups which were allocated to it during said first allocation

[0185] - for each unassigned exploration group of said partitioned subset, determination of a score including:

[0186] - determining a first distance corresponding to the minimum distance said exploration group not assigned to said exploration groups assigned to said drone having the least surface area to explore;

[0187] - determining a second distance corresponding to the minimum distance said exploration group not assigned to said exploration groups assigned to the other drones of said plurality distinct from said drone having the least surface area to explore;

[0188] - determining the score Sc corresponding to the difference between said first and second distances;

[0189] - until the sum of the areas of the convex hulls of said groups exploration groups allocated to each drone is substantially equal or until all said exploration groups resulting from the partitioning are allocated, second allocation of exploration groups not allocated during said first allocation, the drone having the least surface area to explore being allocated the unallocated exploration group with the lowest score.

[0190] According to a second option, as described below in relation to [Fig.6], said allocation comprises the set 78 of the following sub-steps:

[0191] - calculation of the convex hull of all exploration groups of said sub- partitioned set;

[0192] - for each of said drones:

[0193] - determination of the vertex of the convex envelope closest to said drone considered ;

[0194] - determination of the exploration groups of said partitioned subset the most close to the said summit,

[0195] - among said exploration groups of said partitioned subset the most close to said summit, determination of said exploration group closest to said drone considered;

[0196] - allocation of said closest exploration group P_P to said drone considered.

[0197] The choice of one of the allocation options 56 corresponding to the set 76 of sub-steps or to the set 78 of sub-steps then conditions the implementation of step 58 of the order of inter-exploration group (i.e. inter-cluster) traversal of the same list.

[0198] Indeed, according to the first option, corresponding to the set 76 of sub-steps, at least one list assigned to a drone of said plurality of drones is capable of containing several exploration groups so that it is necessary to determine the order of inter-exploration group traversal of this list.

[0199] The route to be determined for the drone in question must be the shortest path starting from its position and joining all the clusters it must visit. This problem is a variant 80 of the well-known Traveling Salesman Problem (TSP) to which the present invention proposes a heuristic solution described below.

[0200] According to this variant 80 implemented in the case of the application of the first allocation option 56, the determination 58 of the order of travel of said exploration groups (and not of the points) therefore comprises the optimization of the travel distance starting at the current position of said drone considered and passing through all of said exploration groups which are allocated to it by solving, by means of a 3-opt algorithm, a problem corresponding substantially to the traveling salesman problem, and by considering a predetermined limit number of allocated exploration groups, said travel order being re-optimized via said 3-opt algorithm as soon as one of its allocated exploration groups is sufficiently explored, by adding the allocated exploration group closest to the last of the allocated exploration groups belonging to the predetermined limit number of exploration groups allocated during the previous iteration of the 3-opt algorithm.

[0201] Indeed, in order to guarantee an efficient route for each drone, the device 10 according to the present invention (also called high-level controller) must find the shortest path starting at the current position of each drone and passing through all of the exploration groups (clusters) which are respectively assigned to them (until all the clusters are assigned to a drone), and this from the first iteration of the following steps 60 and 62.

[0202] This problem being NP-complex (NP meaning "non-deterministic polynomial timed"), it is not possible to solve it exactly on a large number of clusters. Also, the approach used according to the present invention is twofold: solving a partial version of the traveling salesman problem (TSP) and this in an approximate manner. In other words, instead of considering all the exploration groups (i.e. clusters) which will be attributed to the drones from the first iteration of the following steps 60 and 62, only a finite number are considered, for example ten at most, and an acceptable solution to the traveling salesman problem is found, but which has no guarantee of being the best. This solution is determined using the heuristic called "3-opt". This is how the order of traversal of the clusters is determined for each drone at each new calculation of the attributions.As soon as a cluster is marked as observed according to step 60, the 3-opt heuristic is restarted after adding the closest cluster already assigned to the drone, but which was not yet in the list of ten.

[0203] The second allocation option 56 corresponding to the set 78 of sub-steps systematically and only affects the closest free exploration group, so that according to this second option, the determination 58 of an order of inter-exploration group traversal within the same list is S: without object, according to sub-step 82, since each list only comprises a single exploration group.

[0204] The following monitoring step 60 notably comprises the sub-step 84 of detection D according to a predetermined criterion, of a sufficiency of exploration of at least one current allocated exploration group.

[0205] As an optional addition, as illustrated below by [Fig.7], said predetermined criterion depends, for each exploration group, at least on the following elements:

[0206] - the density of pixels to be explored corresponding to the proportion of pixels to be explored, each pixel to be explored corresponding to a point of interest to be explored, among the set of pixels constituting the current exploration group;

[0207] - the exploration rate corresponding to the proportion of pixels already explored among the set of pixels to explore in the current exploration group;

[0208] - the derivative of the exploration rate;

[0209] a plurality of distinct pixel density ranges being defined, each pixel density range being respectively associated with a pair of thresholds comprising a scanning rate threshold and a derivative threshold of said scanning rate;

[0210] for the density range of the current exploration group, the exploration sufficiency being obtained when the exploration rate is greater than said exploration rate threshold of said pair associated with said density range and when the exploration rate is less than the derivative threshold of said exploration rate.

[0211] According to this optional addition, with respect to the classic works of the state of the art, it should be noted that such a predetermined criterion makes it possible to consider as a success an exploration which is not total (if the number of unobserved points remains reasonable, of the order of 2%, or even 10%, in particular if these unobserved points are not connected), whereas in the case of the classic TSP, a non-total exploration, or even revisits of points, is often considered as an invalid solution.

[0212] As an alternative, said predetermined criterion, of sufficient exploration of at least one current assigned exploration group, corresponds to the achievement of a predetermined redundancy threshold, noting that by redundancy, we mean the need to repeat the observation for example from different observation angles according to a predetermined angle difference between each observation angle.

[0213] Finally, as an optional addition, the step 62 of updating M_A_J further comprises a sub-step 86 of reassignment R of said exploration groups remaining to be traveled when the updated list of at least one of said drones is empty, said reassignment corresponding at least to the assignment, to said drone whose list is empty, of at least one exploration group remaining to be traveled assigned to the drone having the most remaining surface area to explore, said at least one reassigned exploration group remaining to be traveled being the furthest away or the last to be visited according to the order of travel of said drone having the most remaining surface area to explore.

[0214] In other words, when a drone has finished exploring all the exploration groups (i.e. clusters) that have been assigned to it, always with a view to equitable distribution of workloads, the device 10 (i.e. high-level controller) will assign to this drone exploration groups (i.e. clusters) belonging to the drones that are furthest behind in their exploration. This makes it possible to balance the workload more finely.

[0215] This update process is performed each time a drone finishes exploring a cluster and is therefore subject to the constraint of running in real time. The choice of the cluster to be reassigned is based on the distance to the drone, the furthest cluster being reassigned or, in the case where the path of the clusters is calculated via a heuristic variant 80 of the TSP, as described previously, the last cluster in the planned visit list is the one that is reassigned.

[0216] [Fig.4] illustrates via the schematic representation 90, the implementation of the third partitioning option 74 via a predetermined tiling heuristic T_H.

[0217] This schematic representation 90 comprises nine views 92, 94, 96, 98, 100, 102, 104, 106 which illustrate the step-by-step tiling of the image, for example 500 pixels by 500 pixels, corresponding to the total surface associated with the current mission of the plurality of drones, each tile corresponding to an exploration group according to the present invention.

[0218] Said heuristic makes it possible to find a result equivalent to tiling by linear programming, without guarantee of optimality on the number of rectangles found but in a very short time, which is compatible with the need to be able to restart the calculations during the mission, following a change in the input hypotheses.

[0219] The approach begins by choosing an arbitrary corner of the mission terrain illustrated by view 92 comprising the points P of interest to be observed, typically the bottom left corner and a first horizontal line L of application of said heuristic.

[0220] A tile Fl is placed in this corner, and is shifted to the right, as long as no pixel corresponding to a point of interest P to be observed inside the tile is outside. Then, this same tile is shifted upwards, always until a pixel is excluded. This first tile then has its final position as illustrated by the position of the tile Fl in view 94.

[0221] The marking of the pixels corresponding to the points of interest P to be observed of the tile F1 is removed from the terrain as illustrated by the following view 96 to determine the position of the following tiles. The second tile F2 is placed at the bottom, on the right of the first tile FL In the same way as the tile F1, the tile F2 is also moved to the right then upwards until its final position is obtained.

[0222] As illustrated by views 98, 100 and 102, the position of the other tiles F3, F4 and F5 is determined in a similar manner until reaching the end of the first row as illustrated by view 102 where all points of interest that were under line L are now covered by a tile.

[0223] As illustrated by views 104 and 106, the determination of the following tiles F6 and F7 is repeated in a similar manner on a terrain where the first row has been truncated, the line L being moved upwards as illustrated by view 104 and so on until the complete tiling (i.e. partitioning) illustrated by view 108 is obtained.

[0224] When such tiling is used with tiles (i.e. clusters or exploration groups according to the present invention) of rectangular or even square shape, it nevertheless remains optionally interesting subsequently to calculate the convex hull of the pixels which are inside and its area. This area can be useful, as indicated previously, in the mechanism for assigning exploration groups (i.e. clusters) to drones.

[0225] It should be noted that according to [Fig.4], a horizontal processing by horizontal row / line is implemented, but that a vertical processing by vertical column / line is deducible directly and unambiguously by the person skilled in the art from this example.

[0226] [Fig.5] illustrates the first allocation option 56 corresponding to an auction system based on the determination and use of a score Sc in order to guarantee that the drones have similar surfaces to explore, that the order of visit is efficient and that the exploration groups (i.e. clusters) allocated to each drone are not scattered to the four corners of the mission area, but rather that they are relatively close to each other. In order to manage this distribution (i.e. allocation), the device 10 according to the present invention (also called high-level controller) has determined / received as input from the allocation module 16, the information concerning the partitioning 54 (i.e. clustering), the positions of each drone.

[0227] More precisely, view 110 of [Fig.5] illustrates the result of the sub-step of the set 76 corresponding to the first allocation, to said plurality of drones represented by the crosses Ai, A2 and A3, of all the exploration groups comprising at least one point of interest to be explored as a priority and / or of the closest exploration group.

[0228] In other words, according to this first view, the drone Ai has received according to this first allocation the mission of exploring the exploration groups Gi i and Gk 2 - the drone A2 has received according to this first allocation the mission of exploring the exploration groups G2 1, G2 2 and G2 3, the drone A3 has received according to this first allocation the mission of exploring the exploration groups G3 _b G3 2 and G3 3.

[0229] As previously indicated, in order to guarantee fairness of the workloads of each drone Ab A2 and A3, it is always the drone which has the least surface area to explorer (sum of the surfaces of the convex hulls of the exploration groups (i.e. clusters)) which is assigned a new exploration group (i.e. cluster).

[0230] According to the first allocation option 56, this choice is based on a score which is calculated for each unassigned exploration group (i.e. cluster). The score used contains two components. The first component evaluates the minimum distance of the unassigned exploration group (i.e. cluster) considered to the exploration groups (i.e. clusters) already assigned to the drone considered. This first component favors the allocation of exploration groups (i.e. clusters) close to those already assigned to minimize the movements made by the drones to reach their next clusters.

[0231] View 112 illustrates this first minimum distance di for the unassigned exploration group 114 to the exploration groups (i.e. clusters) Gi i and Gi_ 2 assigned to the drone Ab, the distance to the exploration group Gi 2 being retained as the minimum distance dh, the distance (not shown) to the exploration group Gi_ being greater.

[0232] The second component evaluates the minimum distance of the cluster considered to the clusters assigned to the other drones. This prevents the drones from crossing each other too much.

[0233] View 116 illustrates this second minimum distance d2 for the unassigned exploration group 114 to the exploration groups (i.e. clusters) G2 1, G2 2 and G2 3 assigned to the drone A2 and to the exploration groups (i.e. clusters) G3 1, G3 2 and G3 3 assigned to the drone A3, the distance to the exploration group G2 1 being retained as the minimum distance d2, the other distances to the other G2 2, G2 3, G3 _b G3 2 and G3 3 being greater.

[0234] These two components are then combined to give the score Sc of each cluster with: Sc = dl - d2.

[0235] The exploration group (i.e. cluster) with the lowest score is then assigned to the drone whose turn it was to be assigned a new exploration group (i.e. cluster), namely the one with the least surface area to explore. This process promotes consistent and more efficient allocations for the drones' path.

[0236] [Fig.6] illustrates the second allocation option 56 corresponding to the implementation of the set 78 of sub-steps of [Fig.3]. This second option aims not to affect all the exploration groups (i.e. clusters) from the start of the mission, but instead to systematically and only affect the closest free cluster P_P of each drone.

[0237] Then, when this second option is implemented during the allocation step 56, during the update step 62, as soon as an exploration group (ie cluster) is marked as observed during step 60, the nearest free exploration group (ie cluster) is again assigned to the drone that has just finished the observation.

[0238] This approach typically involves sometimes leaving one or more clusters aside, before the drones return to them at the end of the mission.

[0239] To circumvent this problem, it is proposed according to this second option to calculate the convex envelope of all the exploration groups (i.e. clusters), and to assign to each drone the nearest vertex of this envelope.

[0240] View 118 of [Fig.6] illustrates the initial situation with the position of the drone Ai and the exploration groups of substantially rectangular or square shape resulting from the partitioning step 54.

[0241] View 120 illustrates the convex hull E_C calculated according to this second option and the nearest vertex 121 assigned to the drone Ab

[0242] The view 122 illustrates the zone 123 comprising the exploration groups (i.e. clusters) comprising points of interest P, closest to said vertex 121 of the view 120, namely the exploration group (i.e. cluster) 124 centered on the pixel 125, the exploration group 126 (i.e. cluster) centered on the pixel 127, the exploration group 128 (i.e. cluster) centered on the pixel 129.

[0243] View 130 illustrates the allocation of the exploration group (i.e. cluster) 124 which among the exploration groups 124, 126 and 128 is the exploration group closest to the drone Ab

[0244] When the next exploration group (i.e. cluster) to be assigned to a drone must be determined, the device 10 selects the closest exploration group (i.e. cluster) only if this does not move said drone in question away from the vertex assigned to it.

[0245] Otherwise, according to this second allocation option 56, the device 10 then selects the closest exploration group (i.e. cluster) which does not distance said drone considered from the vertex (this cluster necessarily exists). Each time a cluster is marked as observed, the convex hull of all the remaining clusters is recalculated, each drone is assigned, according to this second allocation option 56, again the vertex closest to this envelope E_C. In practice, the old envelope and the new one share vertices in common, and for most drones, the old vertex and the new one coincide from a convex hull to the next convex hull calculated upon detection of the current exploration sufficiency of an allocated exploration group.

[0246] [Fig.7] illustrates the pair of thresholds used to detect the exploration sufficiency of a current exploration group.

[0247] Indeed, as indicated previously, the device 10 according to the present invention must detect when it must assign a new exploration group to a drone, typically during at least one of the following events:

[0248] - according to a predetermined criterion, of a sufficiency of exploration of at least one group current exploration allocated;

[0249] - changing an exploration priority of at least one of said points of interest;

[0250] - adding or removing a drone from said plurality (i.e. swarm of drones).

[0251] With regard to the detection of the sufficiency of exploration of at least one exploration group, it is proposed to use for this a threshold on the quantity of pixels explored.

[0252] The individual control of the drones, relating to their geographical position and the orientation of the on-board image sensor, is carried out by a neural network, within the framework of reinforcement learning, as implemented in particular according to patent application FR 2213738 in the name of the Applicant, being effective especially at the start of exploration and less so when it comes to completely finishing the exploration.

[0253] Since the systematic nature of the exploration is not necessarily sought to succeed in the current mission and since instead, it is possible to favor a significantly faster exploration, even if it means leaving pixels corresponding to points of interest unexplored, it is useful to determine the thresholds from which the individual control of the drones loses effectiveness.

[0254] Experimentally, it has been found that these thresholds are dependent on pixel densities. By misuse of language, we call "density" the proportion (i.e. the rate) of pixels to be explored among all the pixels of the current exploration group (i.e. current cluster or current exploration window), each pixel to be explored corresponding to a point of interest to be explored. It is then proposed to use these variable thresholds as a function of the density.

[0255] More precisely, as indicated previously, it is interesting to define two thresholds per pixel density range:

[0256] - the first threshold consists of considering the exploration rate corresponding to the proportion of pixels already explored among all the pixels to be explored in the current exploration group. According to the present invention, it is considered that before this threshold is reached, the drone (i.e. the agent) explores the window efficiently, but that once this threshold is reached, a plateau in the exploration rate may appear.

[0257] - the second threshold consists of considering the derivative of the exploration rate, in order to more precisely detect the plateaus at which exploration progresses much more slowly. In practice, we estimate the derivative of the exploration rate over a succession of decision steps of the drone (i.e. of the agent), for example after twenty decision steps.

[0258] [Fig.7] is an example which illustrates the principle of determining these thresholds. For a given density interval, for example in [Fig.7] for a pixel density of between 30% and 100% (i.e. between 0.3 and 1), a drone of the plurality of drones carries out the exploration of a set of geographical zones whose image has a dimension substantially equal to that of an exploration group (i.e. of an exploration window).

[0259] The average curve of exploration rate per episode is calculated, an “episode” corresponding to the exploration of an area of ​​the set of geographical areas), as is the average curve of the derivative of the exploration rate.

[0260] In [Fig.7], in view 132, the increasing curves 134 and 136 correspond respectively to the evolution of the exploration rate when the exploration is implemented by a drone whose individual control, relating to their geographical position and the orientation of the on-board image sensor, is carried out by a neural network, within the framework of reinforcement learning, such as in particular implemented according to patent application FR 2213738 in the name of the Applicant, or when the exploration is implemented by a drone using an unlearned algorithm for exhaustive exploration of the geographical area considered. Curves 138 and 140 correspond respectively to their derivative.

[0261] The exploration rate thresholds correspond respectively to point 142 for the exploration rate threshold whose value is obtained using the rate value ordinate axis on the left of view 132, and to point 144 for the derivative of this rate whose value is obtained using the derived value ordinate axis on the right of view 132, the abscissa axis corresponding to the number of episodes.

[0262] Thus, according to this option of the present invention, it is considered that the exploration group is explored when the exploration rate is greater than the exploration rate threshold, and the derivative of the exploration rate is less than the derivative threshold.

[0263] The table above corresponds to threshold values ​​for a drone controlled individually via a neural network, in the context of reinforcement learning, as implemented in particular according to patent application FR 2213738 in the name of the Applicant, the neural network being convolutional CNN (from the English Convolutional Neural Networks) and favoring efficiency over exhaustiveness: the exploration implemented by the drone is very fast, but potentially leaves a large part of pixels unexplored. In a convolutional neural network, each neuron of the same layer has exactly the same connection pattern as its neighboring neurons, but at different input positions. The connection pattern is called a convolution kernel or, more often, "kernel" in reference to the corresponding English name, or filter, the convolution acting as a filter. A convolutional network is capable of having several patterns or “kernels” per layer. Pixel Density Crawl Rate Threshold Crawl Rate Derivative Threshold 0 to 0.05 0.5 0.001 0.05 to 0.1 0.6 0.001 0.1 to 0.2 0.7 0.0005 0.2 to 0.3 0.8 0.0003 0.3 to 1.0 0.85 0.0002

[0264] Those skilled in the art will understand that the invention is not limited to the embodiments described, nor to the particular examples of the description, the embodiments and variants mentioned above being suitable for being combined with each other to generate new embodiments of the invention.

[0265] The present invention thus makes it possible, in combination with the individual control of the drones, known as low-level, relating to their geographical position and the orientation of the on-board image sensor, is carried out by a neural network, in the context of reinforcement learning, as implemented in particular according to patent application FR 2213738 in the name of the Applicant, to provide high-level control of coordination of the plurality of drones. Control of the drones of the swarm of drones (i.e. of the plurality of drones) is therefore obtained at two levels: low-level individually of the position and the camera (i.e. image sensor) of each drone, and high-level to coordinate them with each other, allowing scaling up, i.e. having the capacity for almost instantaneous calculations for a large number of points of interest to be observed.

[0266] The high-level control proposed via the method 50 and the device 10 according to the present invention for assigning a mission to each drone of a plurality of drones is advantageously based on a partitioning (i.e. clustering) of a subset of the terrain associated with the mission, namely the only points of interest to be observed, followed by a mechanism for partial assignment of the exploration groups (i.e. clusters) and finally a determination of inter-cluster routes.

[0267] Such prior partitioning (i.e. clustering), which has no impact because it is implemented in mission preparation, or is directly embedded and can be recalculated if the mission area changes, allows the implementation of almost instantaneous calculations during the inference, consequently very fast, of a neural network of the low-level controller such as that implemented in particular according to patent application FR 2213738 in the name of the Applicant.

[0268] Prior partitioning (i.e. clustering) further allows a reasonable resolution of the traveling salesman problem TSP by making it low-dimensional by construction.

[0269] This speed of calculations allows permanent adaptation to the hazards of the mission, such as a variable number of drones. Indeed, it is sufficient to restart the calculations of the device 10 (i.e. the high-level controller) at the moment when the number of drones varies, while being compatible with a prioritization of the space to be observed, and avoiding a permanent exchange of synchronization information between the drones. The solution also works with intermittent synchronizations because each drone is autonomous in its sliding exploration of the exploration groups (sliding window) which are assigned to it.

Claims

1. Claims Method (50) for assigning a mission to each drone of a plurality of drones individually controlled by means of a neural network, each drone being equipped with at least one image sensor whose orientation and / or zoom is controllable, the mission of the plurality of drones being to remotely explore a plurality of predetermined points of interest within a predetermined geographical area, the method being implemented by an electronic device for assigning a mission to each drone of a plurality of drones, the method comprising at least the following steps: - obtaining (52) a digital model of the terrain associated with said predetermined geographical area and the position of each of said points of interest of said plurality within said digital model of said predetermined geographical area, said points of interest forming a subset of said predetermined geographical area,said digital model being an image of said predetermined geographic area, said image comprising a predetermined number of pixels distributed within said image according to a predetermined two-dimensional grid;, - partitioning (54) said subset into exploration groups grouping together neighboring points of interest being spaced from each other by a distance less than a predetermined distance threshold, each group being of substantially equal size; - depending on the current position of each of said drones, allocation (56) to each of said drones of a list of exploration group(s) to be traveled, said list comprising at least one of said exploration groups; - for each of said drones, determination (58) of a route order of said allocated exploration groups when said list comprises at least two of said exploration groups of said partitioned subset; and the following steps implemented iteratively during said mission until said subset is completely explored: - monitoring (60), in parallel, of the position and route of each of said drones, until detection of at least one of the following events: - according to a predetermined criterion, of a sufficient exploration of at least one currently allocated exploration group; - changing an exploration priority of at least one of said points of interest; - adding or removing a drone from said plurality; - at least after each detection, updating (62) the list of exploration group(s) remaining to be covered allocated to each drone by removing from said list any sufficiently explored exploration group, or - upon adding or removing a drone or upon changing the exploration priority, repeating all of the steps starting from the partitioning step (54) applied to the unexplored part of said subset.

2. Method according to claim 1, wherein said partitioning (54) of said subset into exploration groups is a k-means partitioning (70), the optimal number of exploration groups being determined by: - ​​testing in parallel several values ​​of the number of exploration groups, - by summing, for each value of the number of exploration groups, the surface of the convex hull associated with each exploration group, the optimal number corresponding to the value providing the minimum total surface.

3. The method of claim 1, wherein said partitioning (54) of said subset into exploration groups is achieved by linear programming tiling (72).

4. The method of claim 1, wherein said partitioning (54) of said subset into exploration groups is obtained by applying (74) a predetermined tiling heuristic, each exploration group corresponding to a tile of said tiling, the application of said tiling heuristic comprising the following first steps: - placing a first tile of predetermined shape in an arbitrary corner of said image, and - obtaining the final position of said first tile within the image by: - shifting along a first of the directions of the two-dimensional grid as long as no pixel corresponding to one of said points of interest is outside said first tile, then by - shifting along the other of the directions of the two-dimensional grid as long as no pixel corresponding to one of said points of interest is outside said first tile; the points of interest covered by the first tile no longer being taken into account for determining the position of the following tiles; and, until each of said points of interest of said plurality is covered, repeating the following steps: - placing another tile of said predetermined shape: - directly following the tile whose final position was previously obtained along said first direction, or, - along said other direction: - at the start of the line following that of the tile whose final position was previously obtained,if said other direction is horizontal, or - of the column following that of the tile whose final position was previously obtained, if said other direction is horizontal, and - obtaining the final position of said other tile within the image by: - ​​shifting along said first direction as long as no pixel corresponding to one of said points of interest is outside said other tile, then by - shifting along said other direction as long as no pixel corresponding to one of said points of interest is outside said other tile; the points of interest covered by said other tile no longer being taken into account for determining the position of the following tiles.,

5. Method (50) according to any one of the preceding claims, wherein said method further comprises obtaining (68) an exploration priority of at least one of said points of interest, and taking it into account during said allocation, said allocation (56) comprising the set (76) of the following sub-steps:

6. - calculation of the surface of the convex hull associated with each exploration group; - first allocation, to said plurality of drones, of all the exploration groups comprising at least one point of interest to be explored as a priority and / or of the closest exploration group; - determination of the drone having the least surface area to explore corresponding to the drone whose list of exploration group(s) to be covered presents the minimum sum of the surfaces of the convex envelopes of said exploration groups which were allocated to it during said first allocation - for each unassigned exploration group of said partitioned subset, determination of a score comprising: - determining a first distance corresponding to the minimum distance from said exploration group not allocated to said exploration groups allocated to said drone having the least surface area to explore; - determining a second distance corresponding to the minimum distance from said exploration group not allocated to said exploration groups allocated to the other drones of said plurality distinct from said drone having the least surface area to explore; - determining the score corresponding to the difference between said first and second distances; - until the sum of the surfaces of the convex hulls of said exploration groups allocated to each drone is substantially equal or until all said exploration groups resulting from the partitioning are allocated, second allocation of exploration groups not allocated during said first allocation, the drone having the least surface area to explore being allocated the unallocated exploration group with the lowest score. Method (50) according to any one of the preceding claims in which the determination (58) of said travel order comprises the optimization of the travel distance starting at the current position of said drone considered and passing through all of said exploration groups which are assigned to it by solving, by means of a 3-opt algorithm, a problem corresponding substantially to the traveling salesman problem or the multiple traveling salesman problem, and considering a predetermined limit number of assigned exploration groups, said traversal order being re-optimized via said 3-opt algorithm as soon as one of its assigned exploration groups is sufficiently explored, by adding the assigned exploration group closest to the last of the assigned exploration groups belonging to the predetermined limit number of assigned exploration groups during the previous iteration of the 3-opt algorithm.

7. Method according to any one of the preceding claims, wherein said updating further comprises a reassignment of said exploration groups remaining to be traveled when the updated list of at least one of said drones is empty, said reassignment corresponding at least to the assignment, to said drone whose list is empty, of at least one exploration group remaining to be traveled assigned to the drone having the most remaining surface to be explored, said at least one reassigned exploration group remaining to be traveled being the furthest or the last to be visited according to the order of travel of said drone having the most remaining surface to be explored.

8. Method (50) according to any one of the preceding claims 1 to 5 wherein, said allocation (56) comprises the set (78) of the following sub-steps: - calculation of the convex hull of all the exploration groups of said partitioned subset; - for each of said drones: - determination of the vertex of the convex hull closest to said drone considered; - determination of the exploration groups of said partitioned subset closest to said vertex, - among said exploration groups of said partitioned subset closest to said vertex, determination of said exploration group closest to said drone considered; - allocation of said exploration group closest to said drone considered.

9. Method according to any one of the preceding claims in which said predetermined criterion depends, for each exploration group, at least on the following elements: - the density of pixels to be explored corresponding to the proportion of pixels to be explored, each pixel to be explored corresponding to a point

10. of interest to explore, among the set of pixels constituting the current exploration group; - the exploration rate corresponding to the proportion of pixels already explored among all the pixels to be explored in the current exploration group; - the derivative of the exploration rate; a plurality of distinct pixel density ranges being defined, each pixel density range being respectively associated with a pair of thresholds comprising a scanning rate threshold and a derivative threshold of said scanning rate; for the density range of the current exploration group, exploration sufficiency being obtained when the exploration rate is greater than said exploration rate threshold of said pair associated with said density range and when the exploration rate is less than the derivative threshold of said exploration rate. Electronic device (10) for assigning a mission to each drone of a plurality of drones individually controlled by means of a neural network, each drone being provided with at least one image sensor whose orientation and / or zoom is controllable, the mission of the plurality of drones being to remotely explore a plurality of predetermined points of interest within a predetermined geographical area, the electronic device for assigning a mission to each drone of a plurality of drones comprising at least: - an obtaining module (12) configured to obtain a digital model of the terrain associated with said predetermined geographical area and the position of each of said points of interest of said plurality within said digital model of said predetermined geographical area, said points of interest forming a subset of said predetermined geographical area, said digital model being an image of said predetermined geographical area,said image comprising a predetermined number of pixels distributed within said image according to a predetermined two-dimensional grid;, - a partitioning module (14) configured to partition said subset into exploration groups grouping neighboring points of interest spaced from each other by a distance less than a predetermined distance threshold, each group being of substantially equal size; - an allocation module (16) configured to allocate to each of said drones a list of exploration group(s) to be browsed according to the current position of each of said drones, said list comprising at least one of said exploration groups of said partitioned subset; - a determination module (18) configured, for each of said drones, to determine a route order of said allocated exploration groups when said list comprises at least two of said exploration groups; and the following modules, implemented iteratively during said mission until said subset is completely explored: - a tracking module (20) configured to track in parallel the position and the path of each of said drones, until detection of at least one of the following events: - according to a predetermined criterion, of sufficient exploration of at least one current allocated exploration group; - change of an exploration priority of at least one of said points of interest; - adding or removing a drone from said plurality; - an update module (22) configured to update, at least after each detection, the list of exploration group(s) remaining to be explored assigned to each drone by removing from said list any sufficiently explored exploration group.