Method for moving a drone of a drone-in-a-box system and a drone-in-a-box system

Abstract

Method for moving a drone (10) of a drone-in-a-box system (1), comprising the steps of: - returning the drone (10) towards the box (20) at a pre-established height; - making the drone (10) descend towards the landing platform (21); - upon the reaching of a first height from the landing platform (21), aligning the drone (10) with the vertical of the landing platform (21); said aligning step comprising a step of detecting the position of the drone (10) via the GPS module (12) and the position of the landing platform (21) via the GPS reader (22), and a step of comparing the positions to establish whether the drone (10) is aligned with the vertical of the landing platform (21); - if not, possibly adjusting the position of the drone (10) in order to align it; - once aligned, a first step of bringing the drone (10) nearer to the landing platform (21); - upon the reaching of a second height from the landing platform (21), lower than the first, acquiring at least one image of a zone beneath the drone (10) via the camera (11) and applying a vision algorithm to recognise the landing platform (21); - calculating the relative position of the drone (10) with respect thereto and possibly adjusting the position of the drone in order to align it with the vertical of the landing platform (21); - a second step of bringing the drone (10) nearer to the landing platform (21).

Description

[0001] DESCRIPTION METHOD FOR MOVING A DRONE OF A DRONE-IN-A-BOX SYSTEM AND A DRONE-IN-A-BOX SYSTEM

[0002] Technical field

[0003] The present invention relates to a method for moving a drone of a drone-in- a-box system and a drone-in-a-box system.

[0004] Prior art

[0005] As is known, the drone-in-a-box type (as it is internationally defined) is a fully automated drone system integrated into a ground station (box). They perform take-off, flight, landing and recharging of the batteries autonomously without the need for a human pilot on the spot.

[0006] The main advantages of drones-in-a-box are the complete automation of missions, better operational security, the possibility of 24-hour monitoring and the absence of the need for dedicated personnel in the field. The drone can enter action in several ways:

[0007] - Missions planned at pre-established days / times;

[0008] - Take-off in the case of an event (triggering an alarm);

[0009] - Manually.

[0010] Thanks to the possibility of streaming live video, they are particularly suitable for applications which include infrastructure monitoring, security and surveillance, industrial site inspection, mapping, aerial surveillance and much more. Automation makes them ideal for repetitive, on-demand missions.

[0011] Aim of the invention

[0012] In this context, the technical task underpinning the present invention is that of proposing a method for moving a drone of a drone-in-a-box system and a drone-in-a-box system, which obviate the drawbacks of the prior art cited above.

[0013] In particular, the object of the present invention is to propose a method for moving a drone of a drone-in-a-box system and a drone-in-a-box system, which guarantee a precision landing in the respective box even for low- weight drones, without the use of additional instrumentation on board the drone itself.

[0014] The stated technical task and specified objects are substantially achieved by a method for moving a drone of a drone-in-a-box system, said drone comprising a camera configured to acquire images of a zone beneath the drone and a GPS module, said drone-in-a-box system also comprising a box, in turn comprising a landing platform and a GPS reader positioned below the landing platform, said method comprising the steps of:

[0015] - returning the drone towards the box, said returning step taking place by moving the drone at a pre-established height;

[0016] - upon the drone reaching a zone above the box, making the drone descend towards the landing platform;

[0017] - upon the reaching of a first height from the landing platform, aligning the drone with the vertical of the landing platform; said aligning step comprising a step of detecting the position of the drone via the GPS module and the position of the landing platform via the GPS reader, and a step of comparing the position of the drone with the position of the landing platform to establish whether the drone is aligned with the vertical of the landing platform;

[0018] - if not, possibly adjusting the position of the drone based on the comparison made in order to align it with the vertical of the landing platform;

[0019] - once the drone is aligned with the vertical, a first step of bringing the drone nearer to the landing platform;

[0020] - upon the reaching of a second height from the landing platform, lower than the first, acquiring at least one image of a zone beneath the drone via the camera and applying a vision algorithm to recognise the landing platform;

[0021] - calculating the relative position of the drone with respect thereto and possibly adjusting the position of the drone based on the calculation made in order to align it with the vertical of the landing platform;

[0022] - a second step of bringing the drone nearer to the landing platform.

[0023] According to an aspect of the invention, in the event of a failed recognition of the landing platform via the vision algorithm, the method comprises a step of bringing the drone back up to the first height and repeating the step of aligning the drone with the vertical of the landing platform.

[0024] According to an aspect of the invention, after a predefined number of repetitions of the aligning step, the drone is made to land on the ground and simultaneously an error message containing the position of the drone is sent.

[0025] According to an aspect of the invention, upon the reaching of a third height, lower than the second, the drone is made to descend so as to land on the landing platform without further monitoring of the alignment.

[0026] Preferably, the step of acquiring at least one image and the step of applying a vision algorithm are repeated at least a second time upon the reaching of a fourth height from the landing platform, lower than the second height.

[0027] Preferably, during the second step of bringing nearer, the step of acquiring at least one image, the step of applying a vision algorithm and the step of calculating the relative position of the drone with respect to the landing platform are repeated a plurality of times at different heights.

[0028] According to an aspect of the invention, the vision algorithm is executed by means of a previously trained convolutional neural network, said convolutional neural network being configured to recognise the landing platform in response to the acquisition of said at least one image.

[0029] Preferably, the vision algorithm comprises a fine-tuning step wherein, starting from a model pre-trained on a dataset taken at a given height, the model is fine-tuned using specific datasets for a plurality of heights of interest.

[0030] The stated technical task and specified objects are substantially achieved by a drone-in-a-box system, comprising a drone and a box, said drone comprising a camera configured to acquire images of a zone beneath the drone and a GPS module, said box comprising a landing platform and a GPS reader positioned below the landing platform; said system comprising a control unit configured to perform the steps of the method as described. Brief description of the drawings

[0031] Further features and advantages of the present invention will become more apparent from the indicative and thus non-limiting description of a preferred but non-exclusive embodiment of a method for moving a drone of a drone- in-a-box system and a drone-in-a-box system, as illustrated in the attached drawings in which:

[0032] - Figure 1 illustrates a schematic view of a drone-in-a-box system according to the present invention;

[0033] - Figure 2 illustrates a schematic block view of the drone of the drone- in-a-box system of Figure 1 ;

[0034] - Figure 3 illustrates an image from above of the landing platform of the drone-in-a-box system used for training a vision algorithm;

[0035] - Figure 4 illustrates a schematic front view of the drone approaching the landing platform and the variation with the height of the vision cone of the drone camera.

[0036] Detailed description of preferred embodiments of the invention

[0037] With reference to the figures, an object of the present invention is a method for moving a drone 10 of a drone-in-a-box system 1 .

[0038] The drone 10 comprises a camera 11 configured to acquire images of a zone beneath it and a GPS module 12. The system 1 also comprises a box 20. The box 20 in turn comprises a landing platform 21 and a GPS reader 22 positioned below the landing platform 21 . Preferably, the GPS reader 22 is positioned below a central zone of the landing platform 21 .

[0039] The method comprises a step of returning the drone 10 towards the box 20. During this step, the drone 10 is commanded to return towards a reference position, identifying the position of the box 20. For example, the reference position is transmitted to the drone 10 when it is commanded to return towards the box 20.

[0040] During the return step, the drone 10 flies at a pre-established return height. Typically, the height from the ground is chosen so as to be high enough to avoid all the obstacles that could be encountered on the return route, such as trees, pylons, turrets, etc. For example, the return height is about 2025 metres.

[0041] In accordance with an embodiment, the return of the drone 10 towards a reference position takes place by detecting the GPS position of the drone 10 via the GPS module 12 and comparing it with the reference position.

[0042] When the drone 10 is in a zone above the box 20 (i.e. when it reaches the reference position), the method comprises a step of making the drone 10 descend towards the landing platform of the box 20.

[0043] For example, it can be established that the drone 10 is in the reference position when it is within an area of distance from the reference position. In the embodiment described above, this occurs when the detected GPS position corresponds within a pre-established threshold to the reference position.

[0044] During the descent, upon reaching a first height from the platform, the method comprises a step of aligning the drone 10 with the vertical of the landing platform 21. This expression is intended as an imaginary line originating from the landing platform 21 and having vertical extension upwards therefrom (i.e. a line orthogonal to the platform itself).

[0045] The alignment comprises a step of detecting the position of the landing platform 21 via the GPS reader 22 and the position of the drone 10 via the GPS module 12. Thereafter, a step takes place of comparing the position of the drone 10 with the position of the landing platform 21 to determine whether the drone 10 is aligned with the vertical of the landing platform 21 . If not, the method comprises a step of possibly adjusting the position of the drone 10 based on the comparison made in order to align it with the vertical of the landing platform 21 .

[0046] In the prior art systems, there is a typical "Return To Home" feature common to most drones, which works by taking the GPS coordinates of the take-off point as a reference. However, the reading of the GPS coordinates of a fixed point on Earth varies constantly due to the dynamic nature of the GPS system (moving satellites) and the environment in which it operates (reflections and multipath). All this contributes to constantly varying the reading of the position of the platform and it may occur that even after a few minutes the reading of the current position differs by a few metres from the initial reading, thus the drone would find itself landing metres away, convinced that it has positioned itself perfectly in the vertical thereof. The positioning of the precision GPS reader 22 below the platform, which constantly transmits the new coordinates thereof to the drone, allows to overcome this positioning error.

[0047] Once the drone 10 is aligned with the vertical, the method comprises a first step of bringing the drone 10 nearer to the landing platform 21 . Such a first step of bringing nearer occurs between the first height and a second height, lower than the first.

[0048] Upon reaching the second height, the method comprises a step of acquiring at least one image of a zone below the drone 10 via the camera 11. The method comprises a step of applying a vision algorithm to recognise the landing platform 21 and calculate the relative position of the drone 10 with respect thereto.

[0049] Preferably, the method comprises a step of enabling the camera 11 upon reaching the second height. This means that the camera 11 is switched off or in any case does not acquire images during the return step and during the first descent by means of GPS comparison.

[0050] In particular, the vision algorithm which can be used is of the object recognition type (known in the field as "Object Detection").

[0051] When choosing the vision algorithm, the field of application must be taken into account. In fact, the possibility of using the method for drones of any class and in particular for lightweight drones, also implies that a CPU of the associated drone-in-a-box system 1 has limited calculation capacity. This severely limits the choice of algorithms to be used, as well as the training techniques.

[0052] Furthermore, the need for this application to recognise an object at different distances, from the initial 10 metres up to the last and decisive 1.5 metres, the height from which the drone will descend without any further measurement, must also be taken into account.

[0053] In the preferred embodiment, the vision algorithm is executed by means of a previously trained convolutional neural network, said convolutional neural network being configured to recognise the landing platform 21 in response to the acquisition of an image.

[0054] Convolutional neural networks (CNN) are capable of independently learning the relevant characteristics of an image without the need for a manual definition of the features. The image used for training, exemplarily illustrated in Figure 3, is an image of the landing platform, which has the actual dimensions thereof inside the box (50cm x 50cm). CNNs are also capable of recognising objects regardless of their position in the image.

[0055] As mentioned above, there is also the problem of recognising the landing platform at different heights. To overcome this problem, a distance fine- tuning method was chosen, which starts from a model pre-trained on a dataset taken at a chosen height (for example 5 metres in height) and then the model is fine-tuned using specific datasets for a plurality of heights of interest.

[0056] Thereby, it is also possible to take advantage of the transfer of learning and reduce the data needed for training but, as will be clearer below, it is also more effective when transferring the package from the cloud to the edge device.

[0057] In the preferred embodiment, given the need to use the model in an application and not in a cloud system, the TensorFlow framework was chosen, as it allows to transfer the model (trained in Google Cloud Vision) to the application (transfer from Cloud to Edge). Thereby, the best of both worlds is exploited, that is, training in a Cloud environment using all the resources and computing power available and then using the trained object in a poor computing and unconnected Edge environment.

[0058] The final result gives a confidence of more than 85%, sufficient for the application in question. The choice of the second height varies according to the applications and is in any case constrained to the possibility for the vision system to recognise the landing platform 21. An exemplary value of this second height is comprised between 7 and 10 metres.

[0059] Visual recognition has a limit given by the distance (height) at which the drone 10 is located. As can be seen in Figure 4, the higher the drone 10, the easier it is for the landing platform 21 to be included in the vision cone thereof, but also the smaller the relative size of the framed platform. Consequently, there is an indeterminacy in the starting position which is not easily obvious given the fact that if the drone 10 approaches at low heights, it risks the platform not falling into its view; if it instead flies very high, it risks not recognising the platform 21 . For these reasons, the alignment in the first descent step is entrusted to the GPS positioning described and only when the second height is reached, is visual recognition used.

[0060] The method comprises a possible step of adjusting the position of the drone 10 based on the calculated relative position. The objective is to keep the drone 10 aligned with the vertical.

[0061] The method then comprises a second step of bringing the drone 10 nearer to the landing platform. Preferably, such a second step of bringing nearer occurs between the second height and a third height, lower than the second. Preferably, upon reaching the third height, the method comprises a step of making the drone 10 descend while landing on the landing platform 21 . The third height is for example 1.5 metres. From the third height down, no alignment monitoring with the vertical is carried out.

[0062] In summary, the positioning of the drone 10 with the vertical of the landing platform 21 is of particular importance in order to perform a landing. The control of this positioning significantly influences the final landing precision. The method described above thus comes into play, mainly consisting of two steps of bringing the drone 10 nearer to the landing platform 21. The first step aims to align the drone 10 with the vertical of the landing platform 21 by means of a GPS comparison, constantly repeated during the step itself. Once the drone 10 is located on the vertical of the landing platform 21 , it descends vertically to the second height, where the vision algorithm comes into play. The algorithm causes the drone 10 to position itself as much as possible on the vertical of the centre of the platform 21. Thereafter, the drone 10 descends and lands on the platform 21 .

[0063] In accordance with an embodiment, in the event of a failed recognition of the landing platform 21 via the vision algorithm, the method comprises a step of bringing the drone 10 back up to the first height and repeating the step of aligning the drone 10 with the vertical of the landing platform 21 .

[0064] In fact, there are situations in which the platform is outside the vision cone of the drone 10, for example due to wind gusts, light effects, reading error, etc. In this case, the second step of bringing nearer cannot occur as expected, but the drone 10 retries the alignment prior to the first step of bringing nearer, whereby it rises again to the first height and repeats the alignment via GPS.

[0065] Preferably, after a predefined number of repetitions of the alignment step (for example, three), the method comprises a step of lowering the drone 10 to the ground and at the same time sending an error message containing the position coordinates.

[0066] In accordance with an embodiment, the step of acquiring at least one image and the step of applying a vision algorithm are repeated at least a second time upon reaching a fourth height, comprised between the second and the third. By way of example, the second height is 7 metres and the fourth height is 3 metres.

[0067] Preferably, the aforesaid steps occur several times during the second step of bringing nearer. More preferably, the visual recognition of the platform (image acquisition and application of the vision algorithm) occurs constantly during the second step of bringing nearer.

[0068] With reference to the figures, the number 1 indicates a drone-in-a-box system, subject matter of the present invention.

[0069] The system 1 comprises a drone 10. The drone 10 in turn comprises a camera 11 configured to acquire images of a zone beneath the drone and a GPS module 12.

[0070] The system 1 comprises a box 20. The box 20 in turn comprises a landing platform 21 and a GPS reader 22 positioned below the landing platform 21 in the centre.

[0071] The system 1 comprises a control unit 30 configured to perform the steps of the method as described. In particular, the control unit 30 is configured to control the moving drone 10 according to the steps of the method described above.

[0072] The characteristics of the method for moving a drone of a drone-in-a-box system and the drone-in-a-box system emerge clearly from the above description, as do the advantages.

[0073] In particular, the proposed method combining a first step of aligning the drone with the vertical via a constant comparison between the position of the drone and the GPS position of the platform via the reader and a second step of alignment via the visual recognition of the platform allows to perform a descent step of the landing drone with sufficient precision to be able to use any type of drone in a drone-in-a-box configuration. This is highly advantageous, especially for drones of very low weight, for example up to 250 grams, which to date, not being able to use additional integrated instrumentation for weight-related reasons, were not able to be used for these applications. Furthermore, the present invention allows to considerably disseminate this drone-in-a-box technology, since according to the ENAC Regulation (Remotely Piloted Aircraft), it is envisaged that for drones under 250 grams in weight, flight registration or obtaining flight authorisation is not required, nor is it necessary to possess a specific license or booklet for piloting. The possibility of use of this class of drones in drone- in-a-box systems also allows non-certified personnel to use the system itself.

[0074] The proposed method therefore allows any drone to enter the box precisely, safely and completely autonomously, without implementing additional instrumentation on board.

[0075] Furthermore, the use of a machine learning-trained vision algorithm increases the accuracy and reliability of the system.

[0076] Furthermore, the reiteration process of the alignment in the event of non- recognition of the platform via the vision algorithm greatly reduces the positioning error and consequently the incorrect landings, increasing the reliability of the system.

Claims

CLAIMS1. A method for moving a drone (10) of a drone-in-a-box system (1 ), said drone (10) comprising a camera (11 ) configured to acquire images of a zone beneath the drone and a GPS module (12), said drone-in-a-box system (1 ) also comprising a box (20), in turn comprising a landing platform (21 ) and a GPS reader (22) positioned below the landing platform (21 ), said method comprising the steps of:- returning the drone (10) towards the box (20), said returning step taking place by moving the drone (10) at a pre-established height;- upon the drone (10) reaching a zone above the box (20), making the drone (10) descend towards the landing platform (21 );- upon the reaching of a first height from the landing platform (21 ), aligning the drone (10) with the vertical of the landing platform (21 ); said aligning step comprising a step of detecting the position of the drone (10) via the GPS module (12) and the position of the landing platform (21 ) via the GPS reader (22), and a step of comparing the position of the drone (10) with the position of the landing platform (21 ) to establish whether the drone (10) is aligned with the vertical of the landing platform (21 );- if not, adjusting the position of the drone (10) based on the comparison made in order to align it with the vertical of the landing platform (21 );- once the drone (10) is aligned with the vertical, a first step of bringing the drone (10) nearer to the landing platform (21 );- upon the reaching of a second height from the landing platform (21 ), lower than the first, acquiring at least one image of a zone beneath the drone (10) via the camera (11 ) and applying a vision algorithm to recognise the landing platform (21 );- calculating the relative position of the drone (10) with respect thereto and possibly adjusting the position of the drone (10) based on the calculation made in order to align it with the vertical of the landing platform (21 );- a second step of bringing the drone (10) nearer to the landing platform2. The method according to claim 1 , wherein, in the event of a failed recognition of the landing platform (21 ) via the vision algorithm, said method comprises a step of bringing the drone (10) back up to the first height and repeating the step of aligning the drone (10) with the vertical of the landing platform (21 ).

3. The method according to claim 2, wherein after a predefined number of repetitions of the aligning step, the drone (10) is made to land on the ground and simultaneously an error message containing the position of the drone (10) is sent.

4. The method according to any one of the preceding claims, wherein upon the reaching of a third height, lower than the second, the drone (10) is made to descend so as to land on the landing platform (21 ) without further monitoring of the alignment.

5. The method according to any one of the preceding claims, wherein the step of acquiring at least one image and the step of applying a vision algorithm are repeated at least a second time upon the reaching of a fourth height from the landing platform (21 ), lower than the second height.

6. The method according to any one of the preceding claims, wherein, during the second step of bringing nearer, the step of acquiring at least one image, the step of applying a vision algorithm and the step of calculating the relative position of the drone with respect to the landing platform (21 ) are repeated a plurality of times at different heights.

7. The method according to any one of the preceding claims, wherein the vision algorithm is executed by means of a previously trained convolutional neural network, said convolutional neural network being configured to recognise the landing platform (21 ) in response to the acquisition of said at least one image.

8. The method according to any one of the preceding claims, wherein the vision algorithm comprises a fine-tuning step wherein, starting from a model pre-trained on a dataset taken at a given height, the model is fine-tuned using specific datasets for a plurality of heights of interest.

9. A drone-in-a-box system (1 ), comprising a drone (10) and a box (20), said drone (10) comprising a camera (11 ) configured to acquire images of a zone beneath it and a GPS module (12), said box (20) comprising a landing platform (21 ) and a GPS reader (22) positioned below the landing platform (21 ); said system (1 ) comprising a control unit (30) configured to perform the steps of the method according to any one of claims 1 to 8.

10. A computer program comprising instructions which, when carried out by an electronic device, determine the performance of the steps of the method according to any one of claims 1 to 8.

Images