A sensor system for detecting icing conditions in order to avoid potential icing conditions with an unmanned aerial vehicle

Abstract

The present invention relates to a method and arrangement for detecting presence of possible icing conditions of an unmanned aerial vehicle. The method obtains (206) ambient temperature information of the UAV, records (201) at least one diffraction image (25a) comprising information of cloud droplets (25) situated in an open measurement space. The method further transfers (202) the recorded diffraction images (25a) as respective holograms to a computer (3). The method further applies (205) a trained neural network for recognizing the cloud droplets (25), where an input of the trained neural network is the at least one hologram and where an output of the trained neural network comprises particle size information of the cloud droplets (25), and calculating (208) a presence of possible icing conditions based on the obtained ambient temperature information of the UAV and the particle size information of the cloud droplets (25).

Description

[0001] A SENSOR SYSTEM FOR DETECTING ICING CONDITIONS IN ORDER TO AVOID POTENTIAL ICING CONDITIONS WITH AN UNMANNED AERIAL VEHICLE

[0002] Technical field

[0003] The present invention relates to unmanned aerial vehicles (i.e. UAVs), such as drones, and their capability to fly safely even in freezing conditions.

[0004] Background

[0005] Unmanned aerial vehicles have become increasingly common in various different use situations in latest years. As the air temperature depend notably on the altitude (i.e. vertical distance from the ground), there is also notable temperature variations which an UAV experiences in different phases of a flight. This is especially true in higher altitudes, and with longer flight ranges. The atmosphere comprises notable amounts of water vapor, and when these particles collide with moving or rotating physical structures, there is a possibility for icing on the physical structure of an UAV (e.g. drone). When the icing is extensive enough for instance on the rotating wings of a drone, there is a risk for the drone to fall down on the ground unexpectedly. This poses a safety risk to general public, too, in addition to material losses and related costs of reparation.

[0006] Hence, there is a need for analysing the icing conditions during the flight of the UAV, and to warn the user (i.e. the person handling and controlling the drone, for instance) when there is an immediate danger of malfunctioning of the UAV due to icing on the physical structure of the UAV. A malfunctioning drone may fall down to the ground, and hence, pose a risk in form of human and material damages.

[0007] Patent application publication WO 2018 / 220268 (“Kaikkonen”) discloses a measurement system for establishing a forecast for structural icing of a mechanical structure. There is a digital in-line holography apparatus with a monochromatic light source and a detector which senses diffraction images due to cloud droplets. Sizes and number of cloud droplets are calculated, and further, a liquid water content and median volume diameter of the cloud droplets within the open measurement space are calculated. The calculation logic combines the liquid water content, the median volume diameter, meteorological measurement data (temperature and wind speed) from a weather station, speed of the moving mechanical structure and 3D model information of the mechanical structure for making a forecast about speed of structural icing of the moving mechanical structure. The possible moving mechanical structures may be rotor blades of wind turbines, power transmission lines or wings of an airplane.

[0008] Scientific publication “UAV Icing: A Survey of Recent Developments in Ice Detection Methods”, Low-Hansen et al., Department of Engineering Cybernetics, NTNll & IIBIQ Aerospace, Trondheim, Norway, 2023, discloses detection methods for in-flight icing of uncrewed aerial vehicles. This publication concentrates on direct and indirect methods of detecting the already accumulated solid ice on the structures of the flying vehicle. This reference performs the direct ice detection (for ice present on the wings and the body of the aircraft) applying impedance and capacitance sensors, vibration sensors, heat of fusion sensors, rotating wheel torque sensors, ultrasonic guided waves sensor or time domain reflectometry sensors, or by using so-called pitot tubes or with a so-called fiber-optical method. A weather radar may be used for detecting atmospheric icing conditions. Various models may be applied in indirect performance degradation detection, such as motion, icing effect, aircraft performance or wind models.

[0009] A problem in prior art is that sensor dimensions are relatively large (equalling 1 m in two of the three dimensions / 2016 and tens of centimetres / 2022), the weight of the sensor is large (8 kg I 2016; 800 g I 2022), and the power consumption is relatively large, too. Also, in prior art, the calculation has required the use of a separate PC and a graphics processing unit.

[0010] A related problem in the use of drone technology is that the imaging sensors which are used for e.g. taking photographs or transmitting a laser-based detection signal, need to be located inside a transparent window structure which should not freeze into a non-transparent frosted-kind of a window, when the drone flies into a freezing cloud. On the other hand, if the freezing occurs and the window threatens to be matte (i.e. non-transparent due to ice), there is a problem to get rid of such icing from the window as soon as possible. A further drawback in many prior art solutions has been that weather forecast information forms the basis in many of them, which still have a non-sufficient resolution when concerning locally accurate information relating to local weather and proneness to icing in that specific location. Therefore, the local weather forecast as such is not the best starting point for these kinds of analysis, as the local weather may vary a lot in even a few hundred meters distance, e.g. in the case of local rain showers during the summertime.

[0011] Summary

[0012] An object of the invention is to provide a new and more advanced method and arrangement for obtaining information on icing conditions around an unmanned aerial vehicle (i.e. UAV) during its flight time in the air. The presented new method obtains more accurate information on the risk of icing, which in turn increases safe flying times in the air, and on the other hand, prevents dangerous malfunctioning of the wings of the UAV in the air, which would result in the UAV falling down causing hazards on humans and physical devices and materials alike. The presented new arrangement requires less volumetric space and also less power, and it is lighter than the prior art solutions, so therefore the presented new solution is also cost-effective.

[0013] According to a first aspect of the present invention, there is introduced a method for detecting presence of possible icing conditions of an unmanned aerial vehicle, UAV. The method is characterized in that the method comprises the steps of:

[0014] - obtaining ambient temperature information of the UAV,

[0015] - recording at least one diffraction image comprising information of cloud droplets situated in an open measurement space by an image detector of a digital in-line holography apparatus,

[0016] - transferring the at least one recorded diffraction image as respective holograms to a computer, wherein the computer comprises a processor, - applying, in the processor, a trained neural network for recognizing the cloud droplets, where an input of the trained neural network is the at least one hologram and where an output of the trained neural network comprises particle size information of the cloud droplets, and - calculating, in the processor, a presence of possible icing conditions based on the obtained ambient temperature information of the UAV and the particle size information of the cloud droplets.

[0017] In an embodiment of the presented method, preprocessing the hologram by performing a background removal for the hologram, the background comprising fixed objects shown in the diffraction image.

[0018] In an embodiment of the presented method, preprocessing the hologram by changing a size of the diffraction image.

[0019] In an embodiment of the presented method, providing information from the UAV, comprising information on whether propellers are turned on or off, to the processor.

[0020] In an embodiment of the presented method, providing motor power and / or rotation speed of propellers and / or size of the propellers from the UAV to the processor.

[0021] In an embodiment of the presented method, providing air speed of the UAV to the processor.

[0022] In an embodiment of the presented method, GPS data of the UAV is provided to the processor in order to localize the obtained information, and for allowing the distribution of the obtained information also to other devices, servers and / or to other interested users and / or parties and / or to other flying unmanned or manned vehicles or devices.

[0023] In an embodiment of the presented method, the localized information on possible icing conditions are applied in assisting in making a localized weather forecast.

[0024] In an embodiment of the presented method, if the icing conditions are detected and calculated to be present, also the magnitude of the icing conditions are determined. In an embodiment of the presented method, determining, from the calculated presence of possible icing conditions, and possibly also from its magnitude, an indication on whether it is safe to fly the UAV from such an inspection moment onwards.

[0025] In an embodiment of the presented method, showing a safety parameter of the flying conditions for the UAV as a numeric value or as a traffic lights style indicator to a user or a manager of the UAV.

[0026] In an embodiment of the presented method, the trained neural network is configured to recognize liquid water droplets and solid ice crystals in the hologram and to separate liquid and solid particles in the analysis.

[0027] In an embodiment of the presented method, the method comprises the step of:

[0028] starting the recording step only after the sensed temperature reaches a value below a threshold value, where the threshold value has been set within the temperature range of +2 … +5 °C.

[0029] In an embodiment of the presented method, the image detector is placed in a closed volume where the closed volume is restricted by a window or by double windows locating opposite of one another, where a heating element is placed within the closed volume for managing the temperature within the closed volume and on the surfaces of the window(s).

[0030] In an embodiment of the presented method, the operation and the switching on / off the heating element is based on the measured temperature, wherein further, the heating element is switched on when liquid droplets have been detected and the measured temperature locates between the temperature range of 0 … +2 °C.

[0031] In an embodiment of the presented method, the ambient temperature information of the UAV is obtained by sensing temperature by a temperature sensor comprised within the UAV or as fixed to the UAV.

[0032] In an embodiment of the presented method, the ambient temperature information of the UAV is obtained by using an infrared camera fixed to the UAV. In an embodiment of the presented method, the ambient temperature information of the UAV is obtained by receiving data from an external data source or server comprising a local weather forecast.

[0033] In an embodiment of the presented method, the method further comprises the steps of:

[0034] summing information of at least two consecutive diffraction images for increasing a number of liquid droplets within a single, summed, image; and using the summed image in the calculations performed by the processor, noting the number of consecutive summed images in the calculations as well.

[0035] According to a second aspect of the present invention, there is introduced an arrangement for detecting presence of possible icing conditions of an unmanned aerial vehicle, UAV. The arrangement is characterized in that the arrangement comprises:

[0036] - temperature information collection means configured to obtain ambient temperature information of the UAV,

[0037] - an image detector of a digital in-line holography apparatus configured to record at least one diffraction image comprising information of cloud droplets situated in an open measurement space,

[0038] - a computer, where the at least one recorded diffraction image is / are transferred to as respective holograms, wherein the computer comprises a processor,

[0039] - the processor configured to apply a trained neural network for recognizing the cloud droplets, where an input of the trained neural network is the at least one hologram and where an output of the trained neural network comprises particle size information of the cloud droplets, and

[0040] - the processor configured to calculate a presence of possible icing conditions based on the obtained ambient temperature information of the UAV and the particle size information of the cloud droplets.

[0041] Various embodiments of the method according to the first aspect of the present invention may be implemented as an embodiment of the arrangement, respectively. A third aspect of the present invention is a computer program, and a respective computer readable medium. The computer program comprises computer readable code. When executed in a processor or controller, the computer program may run all applicable method steps described in connection with the first aspect of the present invention.

[0042] Brief description of the drawings

[0043] FIG. 1 illustrates some functional elements of an exemplary measurement arrangement according to the present invention, and

[0044] FIG. 2 illustrates an example of the method according to the present invention in a form of a flow chart.

[0045] Detailed description

[0046] The present invention introduces a new and advanced method, and a respective arrangement, for detecting the presence and magnitude of possible icing conditions on an unmanned aerial vehicle (i.e. UAV) during its flight. The respective detection result(s) may be indicated to the user with a numeric value, or with a traffic-lights style of coloured LED light information, for instance.

[0047] Different embodiments, details and functionalities presented in connection with the method, are applicable also with the respective arrangement i.e. with the respective apparatus. On the other hand, different embodiments, details and functionalities presented in connection with the arrangement, are applicable also with the respective method. A computer program or several computer programs may be applied to realize various method steps, when the respective computer program is executed in a processor or controller of any described data processing device, i.e. a PC or a server or a processor within the UAV itself, for instance. This applies e.g. to any image recording (together with its initiation) and calculation phase, and analysis of an obtained data set, comprising also the activities performed by a trained neural network, and its initiation, for instance. In other words, any described method step which is applicable to be performed or initiated by a computer program, may also be implemented in such a manner. At first, FIG. 1 is referred to. An exemplary icing circumstances forecasting measurement system 100 (i.e. the arrangement according to the invention) is presented where the target of the measurement is to detect and analyze particles in a measurement volume, where the particles may be liquid droplets of water, or some solid matter particles flying in the air.

[0048] In various embodiments, the present invention comprises an in-line holographic measurement apparatus 1 (i.e. a digital in-line holography apparatus), which is now in the focus of FIG. 1.

[0049] In FIG. 1, the icing circumstances forecasting measurement system 100 according to the present invention advantageously comprises an in-line holographic measurement apparatus 1, a computer or processor 3 that has a wired or wireless connection 24 to the in-line holographic measurement apparatus 1. Via this connection 24 the computer or processor 3 may advantageously control operations of the in-line holographic apparatus 1 and also store diffraction images of the cloud droplets that the in-line holographic apparatus 1 has recorded. In an embodiment, the computer 3 may be a circuit card computer.

[0050] The presented icing circumstances forecasting method and system (i.e. arrangement) may advantageously be utilized in the context of airplane wings, such as wings of drones (i.e. UAVs) and other physical parts of the housing of the drone. However, the present invention is not restricted to merely those uses, but instead, the present invention may be applied to any structures which are prone to icing, e.g. electrical equipment on the top of masts, etc. Furthermore, some meteorological station application areas are possible in connection with roadside weather stations, for instance, as the present invention is able to make a forecast on the possible icing conditions currently and in the immediate future.

[0051] In an embodiment of the present invention, the computer or processor 3 belonging to the icing forecasting measurement system 100 has a wired or wireless connection to Internet. Via Internet the computer or processor 3 advantageously has a connection to a server, in such an embodiment. The server may belong to a service provider. The server may have a connection to a database. The database may, in an embodiment, disclose material information and 3D-structural information of different mechanical structures. In an embodiment, it is possible to use the structure shown in FIG. 1 for performing an ice accumulation forecast happening on the wing or wings of the UAV.

[0052] In the icing circumstances forecasting method according to the invention either the computer or processor 3 or the server may process the diffraction pictures of the in-line holographic apparatus 1 by a neural network.

[0053] The in-line holographic measurement apparatus 1 according to an embodiment of the present invention functions without any optical lens system when it records diffraction images of the cloud droplets. This simplifies the structure making the recording of the images in such an embodiment.

[0054] In the illustrated embodiment, the in-line holographic measurement apparatus 1 comprises a coherent electromagnetic radiation part 10 and an image sensor part 20. Both the radiation part 10 and the image sensor part 20 have on one side a window 13 or 23 that is transparent to the utilized monochromatic electromagnetic radiation. The in-line holographic measurement apparatus 1 may also comprise a heating system (not shown in FIG. 1) that may be controlled by utilizing current cloud droplet measurement results.

[0055] The electromagnetic radiation part 10 comprises a point-like wide-angle monochromatic radiation source 12. By using the point-like monochromatic wide-angle radiation source 12, a magnification of a geometric diffraction pattern of cloud droplets may be obtained on the matrix detector 22. Therefore, in the in-line holographic measurement apparatus 1, a magnification of the diffraction patterns of the cloud droplets in the measurement space between the coherent electromagnetic radiation part 10 and the image sensor part 20 may be achieved with a lens-less imaging. This simplifies the required equipment, which is an advantage. A distance between the coherent electromagnetic radiation part 10 and the image sensor part 20 may advantageously be from 20 mm to 100 mm depending on the size of the utilized image sensor. A distance between the windows 13 and 23 may then be between 10 to 20 mm, for example. This distance defines the measurement space, where the particles, especially liquid droplets of water, are of particular interest. The point-like monochromatic radiation source 12 is advantageously a laser, a LED or a spatially filtered broadband light source. Functioning of the point¬ like monochromatic radiation source 12 is advantageously controlled by a control unit 11. The control unit 11 advantageously comprises a power unit. The power unit may be a battery or a power supply having a connection to an electrical network. The control unit 11 advantageously comprises also a processor unit and a memory where the computer programs needed to execute holographic measurement operations according to the present invention may be stored.

[0056] The control unit 11 advantageously comprises also an interface by which a connection 22a to the matrix detector 22 may be accomplished. Via this connection 22a the matrix detector 22 advantageously sends commands, which controls on-off times of the point-like monochromatic radiation source 12. The matrix detector 22 may, for example, command the point-like monochromatic radiation source 12 to transmit radiation advantageously in pulsed mode.

[0057] There may be more than one pulse per one diffraction image, in an embodiment of the invention. The length of the measurement pulse is advantageously less than 50 nanoseconds in order to stop movement of the droplets in an image, in practice.

[0058] In the example of FIG. 1, from the point-like monochromatic radiation source 12 there extends a conical monochromatic radiation pattern 12a towards the matrix detector 22 of the image sensor part 20. The wavelength of the utilized monochromatic radiation may be from 400 nm to 1100 nm. In the conical monochromatic radiation pattern 12a, there can be seen three exemplary cloud droplets 25, 26 and 27. The size of the cloud droplets may vary from 1 μm to 200 μm, in usual atmospheric conditions.

[0059] In the example of FIG. 1, in the conical monochromatic radiation pattern 12a there is an exemplary cloud droplet 25 that scatters and diffracts a part of the monochromatic radiation pattern 12a so that the matrix detector 22 can record an enlarged diffraction image 25a of the cloud droplet 25. Diffraction images of cloud droplets 26 and 27 have not been drawn in FIG. 1 for clarity reasons. Via the connection 24 the recorded diffraction images are transferred from the matrix detector 22 to the computer or processor 3.

[0060] The computer or processor 3 may also control several operations of the in-line holographic apparatus 1. Some examples are commands concerning a start- up, shutdown and measurement parameters settings such as frame rate, luminous intensity, hologram recording mode and heating control, for instance.

[0061] In an embodiment, the temperature is measured continuously with the temperature sensor, and whenever the measured temperature is above a specific threshold value, the cloud droplet measurement is not activated. Only when the measured temperature drops below the specific threshold value, the cloud droplet measurement is switched on. In an embodiment, the specific threshold value is set to a temperature locating within the range of +2 … +5 °C. In other words, it is not reasonable to activate the cloud droplet measurement if the temperature is notably over 0 °C in the flying area of the UAV.

[0062] In case of the embodiment where ice accumulation (meaning: the speed of the accumulation of ice) on the surface of the wing(s) is modelled, the airspeed of the UAV is needed to be known. This piece of data may be obtained e.g. from the controller of the UAV. Another possible option is to measure relative velocity of cloud droplets according to the following principle.

[0063] In the present invention and in the described embodiment on the ice accumulation modelling, it is not necessarily needed to make wind measurements separately. In this embodiment, it is however possible to measure relative velocity of cloud droplets if more than two exposure pulses per image (i.e. per hologram) are used. If a time between the radiation pulses is not kept constant and there are at least 3 radiation pulses per image, for example from 2 to 8 measurement pulses, then a movement direction and relative velocity of the cloud droplets may be determined in the measurement volume, i.e. a wind direction and force may be defined from the stored hologram images.

[0064] Furthermore, in this embodiment, the mechanical 3D-structure and material data of the drone’s wings may be stored in the database. The computer or processor 3 has a communication connection to the server and also to the database, via internet connection.

[0065] In one advantageous embodiment, the computer or processor 3 advantageously preprocesses received diffraction images of the cloud droplets 25, 26 and 27. The computer or processor 3 may for example remove blank images and background from saved diffraction images, add a timestamp and / or group the diffraction images together. After the preprocessing phase, the computer or processor 3 may send the pre-processed diffraction images, possibly meteorological data and also possibly data describing 3D-shape and materials of the drone's wings to the server.

[0066] In the present invention, a trained neural network is capable to produce directly a silhouette image of the droplets together with other objects and particles present within the measurement volume. The various objects are visible in the silhouette image in a focused i.e. clearly outlined manner.

[0067] In an embodiment, the neural network applies an Al-based algorithm for recognizing the various liquid and solid anomalies i.e. objects within the measurement volume i.e. within the open measurement space.

[0068] When the diameter of each cloud droplet 25, 26 and 27 is determined, particle size information and / or the distribution of the sizes among noted liquid particles can be obtained. In an embodiment, also the volume of each cloud droplet may be calculated as a volume of a sphere (however, this is not a mandatory step); then the process may assume that the droplet floating in the air as a cloud or fog droplet approximately has a spherical shape. On the other hand, it may be assumed that a droplet has a distinctive “water droplet shape”, when it is a falling droplet part of rain. A total water amount in grams may be obtained in such an embodiment, when diameters of all cloud droplets 25, 26 and 27 in the open measurement space are known. When this calculated value is divided by the volume of the open measurement space of the in-line holographic measurement apparatus 1, the calculation result depicts water volume versus the open measurement space volume, i.e. liquid water content (LWC) in g / m3in the open measurement space of the in-line holographic measurement apparatus 1. As mentioned above, the calculation of the LWC is an optional step in the described process. When the computer or processor 3 or server (whichever performs the calculations) combines the particle size information with the temperature measurement data of the UAV, the computer or processor 3 or server is capable to detect the presence and / or magnitude of possible icing conditions on the UAV. There may be some additional input parameters for the calculation logic, but they are not mandatory.

[0069] FIG. 2 illustrates an embodiment of the method according to the present invention as a flow chart. Some steps are optional, and they are shown with dashed lines between the related blocks.

[0070] The method may be implemented with an apparatus (i.e. an arrangement) fixed with or attached to the unmanned aerial vehicle (i.e. UAV), such as to a drone.

[0071] The method may apply a point source laser or other respective coherent light source. LEDs and / or lenses may also be used, the latter e.g. for collimating purposes of the light. Hence, in other words, the digital in-line holography apparatus 1 may comprise any light source, with various different possible forms and types, involving lasers and / or LEDs. Furthermore, the apparatus comprises a computer, which further comprises a processor and a memory. Furthermore, an image sensor i.e. a camera for taking the desired images, is required. A temperature sensor is also part of the apparatus. Furthermore, in an embodiment, also a GPS receiver may be comprised in the apparatus fixed to the drone, for positioning the drone accurately during its flight and during different flying conditions.

[0072] The illustrated exemplary procedure 200 starts by recording 201 at least one diffraction image comprising information of cloud droplets situated in an open measurement space by an image detector of a digital in-line holography apparatus, involving a light source.

[0073] Secondly, the detected and recorded results i.e. the at least one recorded diffraction image is / are transferred 202 as respective holograms to a computer. The computer comprises a processor, which executes the following calculations and analysis. There is an optional step, where a raw hologram is created 203 (not shown). The raw hologram means the raw image data comprising e.g. a matrix of intensity values of the diffracted light in different parts (i.e. pixels) of the taken image. This may be obtained directly from the camera i.e. the image sensor.

[0074] In an embodiment, it is possible to normalize the pixel values to so-called float values, before the data is fed to block 205. This normalization step is not shown in FIG. 2.

[0075] Another optional step is the pre-processing 204 of the hologram or the raw hologram. The pre-processing may comprise performing a background removal for the hologram or the raw hologram. Usually, this means removing the stationary objects from the taken image, and they usually comprise dirt or dust particles on the windows of the image detector and also the stuck water droplets (or solid ice pieces) on the windows. After the pre-processing 204, the resulting image is a cleaned image illustrating only freely floating droplets in the open measurement space. This step improves the quality of the performed analysis, as no wrong interpretations are made, for example, for dirt particles.

[0076] Thereafter, as a next step, a trained neural network is applied 205 for the input hologram(s) (either pre-processed or not) for recognizing the cloud droplets. In an embodiment, the trained neural network may be used to recognize and mark, which of the remaining objects are indeed liquid droplets, and which are something else, e.g. solid ice crystals. The neural network may form the images comprising the particles and based on e.g. some additional assumption or parametric value, the neural network is used to determine particle size information for the observed cloud droplets, taking into account also the possible assessment result on liquid droplets and other found objects.

[0077] In other words, the trained neural network outputs the particle size information of the cloud droplets. In the present example, this piece of data is enough to proceed with the algorithm further.

[0078] As a separate step performed with the temperature sensor fixed with the UAV, the current ambient temperature right around the UAV is measured (i.e. detected) 206. In other words, the ambient temperature information of the UAV is obtained, as a separate important parameter. In an embodiment, the temperature measurement may be specified as a first step of the performed method. When the measured temperature reaches a certain temperature threshold or reaches a certain temperature range during the flight of the UAV, it may be determined that the rest of the described process is initiated, for obtaining the presence of possible icing conditions. In other words, the calculation of the risk of the possible icing is calculated only when it is in the first place possible based on the detected ambient temperature. E.g. in summer conditions, and in low flying altitudes, the risk of icing is negligible, and the whole calculation process needs not to be initiated at that kind of warm circumstances. However, this is a matter of selection, and the calculation process may be turned always on, when the detected temperature decreases below a certain threshold value. Some examples of the temperature thresholds are described elsewhere in this description.

[0079] In an embodiment, the ambient temperature is measured continuously, or with predefined measurement instants during the flight. Then when the temperature condition is fulfilled for the risk of the icing, the rest of the calculations will be commenced.

[0080] Furthermore, as an option (in other words, in an embodiment), further additional parameters may be added 207 into the determination logic of the detection of the presence of possible icing conditions. Such additional parameters may comprise at least one of the following: The on / off status of the propellers (i.e. wings) of the UAV, the motor output power, the rotational speed and / or the size of the propellers, the air speed of the UAV, and / or the position data (X, Y, Z) of the UAV obtained from the GPS receiver (an optional element). All these parameters are fully optional, but they may be selected to be used based on a specific requirement, e.g. based on a need for knowing an exact location of the UAV. The other optional parameters may give more accurate results, but they of course make the calculation process a bit more complex. The optional parameters may be switched on by the manager or the user of the UAV working e.g. on the server.

[0081] Finally, as a result of the determination logic (or procedure 200), the processor 3 or the server is executed to calculate 208 a presence of possible icing conditions based on the obtained ambient temperature information of the UAV (from block 206) and the particle size information of the cloud droplets (from block 205, obtained as output from the analysis made by the trained neural network).

[0082] In an embodiment, an icing condition may be obtained and shown either as a numeric value or parameter, or it could be shown as a visual traffic-lights styled LED light information, or with some other appropriate communication means which may be shown to a user or to a manager of the UAV, or any other interested observer handling the flight of the UAV, via the used computer 3 or server or even to a smartphone used by such a user or manager e.g. via a smartphone app. This visualizes the icing conditions immediately to the user of the drone.

[0083] The icing condition may be showing, for instance, an imminent risk for icing for the UAV, which means that there is an immediate risk that the drone will fall down in the short foreseeable future. This may be indicated with a red light to the user or the manager of the UAV.

[0084] In an embodiment, the obtained icing condition may be distributed (i.e. communicated) wirelessly to other UAVs which are equipped with appropriate transmission / reception means, and which locate in close vicinity of the UAV which performed the above-described determination. In an embodiment, the obtained data could be provided to other airborne entities, such as to helicopters in various different usage situations and UAVs used in the military applications. The connecting airborne entity may fly approximately on the similar height (meaning: around a certain range of height, vertically) as common drones, which makes the data transfer feasible. As a further option, the obtained icing condition may be distributed (i.e. communicated) wirelessly to a ground station, which may simply be a PC operated by the manager (i.e. user) of the UAV. As a yet further option, the obtained icing condition may be distributed (i.e. communicated) to a cloud-based storage location. Furthermore, in meteorology, used weather models may be improved by using the obtained liquid droplet and icing condition data, when they are combined with precise location and time data (in practice, using a clock and a GPS receiver). In this sense, the local weather forecasts may be improved notably around the area where the present invention is implemented with a drone or a group of drones or other airborne vehicles. In an embodiment of the present invention, the following process may be implemented for optimizing the number of imaged cloud droplets in the measured image area (i.e. in the open measurement space) to be in a certain optimal range for further processing.

[0085] There has been noted an issue in the described measurement system, when using a certain rate in taking of the photographs of the measurement space. If the rate is one photograph taken in a second, the resulting number of detected water droplets is too small for further processing in the described method. One solution could be increasing the size of the camera sensor, which would increase the quality and the photosensitivity (i.e. sensitivity to light). In a dense cloud, there are approximately 100 pieces of water droplets in one cubic centimetre.

[0086] However, in UAV use situations, the camera sensor needs to preferably be a rather small one for practical purposes. From that dimensional preference, there emerges a challenge on how to obtain a good-quality image with enough droplet data in a relatively quick manner.

[0087] If a camera and a laser are set to take photographs e.g. 10-30 frames per second (i.e. fps), their individual calculations with the help of a neural network would require notably more calculation power.

[0088] For relieving the calculation task, and also for increasing the volume of the measurement space in the sampling process, the present invention may apply a summing of the holograms, in an embodiment of the present invention. Such a summing process means that the image taking rate may be increased from 1 fps to a notably quicker rate, and several taken images (i.e. consecutive images) may be summed to form a single summed image comprising all the detected droplets in all the summed consecutive images. When the summed image is fed to the neural network for performing an analysis, the requirement for calculation power is smaller, and still a larger volume of the measurement space will be covered. There is a certain upper limit for the presented summing of the images, because the summed image may be too crowded i.e. too filled with the cloud droplets, so that the single droplets cannot be anymore distinguished in the summed image. Therefore, in case of a too crammed image in sense of the droplet information, the present invention may decrease the number of the images to be summed, so that the measurement sensitivity increases once again.

[0089] Although it is mentioned that the number of droplets available in a single image may become an issue, the above-mentioned summing is still presented as an optional embodiment within the present invention. The system works also without the above-described summing of consecutive holograms i.e. images.

[0090] Another topic relevant in the measurement environment is the possible freezing of the windows surrounding the sensors, such as the camera sensor i.e. the image sensor. In case of solid ice accumulation on the window(s), the visibility for the camera sensor will suffer greatly, and the quality of the taken images will deteriorate quickly. A solution in an embodiment of the present invention is to use windows in each relevant interface between the sensor area and the outer atmosphere. In other words, the image sensor(s) and other possible sensors may locate in a protected compartment surrounded by a window. Furthermore, there could be a double-window structure, where the image sensor locates between them in a closed compartment. A heating element may be set between the double walls, or within otherwise closed compartment. By applying a rather moderate heating power to the heating element, the temperature between the double walls (i.e. in the volumetric space restricted between the windows and other surrounding walls) can be set to be a bit higher value than the environmental temperature. In this way, no condensation will happen which is an advantage of such an embodiment. It is also possible to adjust the heating power of the heating element so that the water droplets and ice crystals hitting the window(s) will evaporate away from the glass surface(s) quickly. A further effect of the window or the double windows is that the heat generated from the heating element will not propagate towards the electronics present in the device, but instead, it solely will heat the closed air volume between the windows or within otherwise closed compartment accommodating the applied sensor. This increases the working reliability of the electronics of the device, too, which is an added advantage.

[0091] In an embodiment of the present invention, the controlling of the power of the heating element is made based on the sensed measurement results. This means that for instance, when there have been detected some water droplets (meaning that the UAV is proceeding into a cloud), and simultaneously the temperature has been sensed to be between 0... 2 degrees Celsius, the system turns on the power to the heating element in such an embodiment. In other words, when the system notices a freezing cloud, the heating element may be turned on. In a respective manner, when the system notes that there is no more imminent danger of freezing, the system may turn the heating element off.

[0092] In an embodiment, the image detector may have dimensions of 2 cm x 2 cm x 5 cm. In an embodiment, the weight of the sensor is in the range of 40 – 50 g. In an embodiment, the power consumption of the measurement device is approximately 5 W. In various embodiments of the present invention, the required calculations may be performed by a controller embodied in a circuit card form. Hence, no separate PC is required anymore necessarily for the calculations. Still, the use of a separate PC or server is not ruled out, as described above in various different embodiments of the present invention. As a conclusion, the present invention with the above intrinsic characteristics provides a smaller and lighter device which consumes also less power, thus having less and mitigated negative effects on the actual flying capabilities and properties of the UAV.

[0093] The present invention is determined by the appended claims.

Claims

Claims1. A method for detecting presence of possible icing conditions of an unmanned aerial vehicle, UAV, characterized in that the method comprises the steps of:- obtaining (206) ambient temperature information of the UAV,- recording (201) at least one diffraction image (25a) comprising information of cloud droplets (25) situated in an open measurement space by an image detector (22) of a digital in-line holography apparatus (1),- transferring (202) the at least one recorded diffraction image (25a) as respective holograms to a computer (3), wherein the computer (3) comprises a processor,- applying (205), in the processor, a trained neural network for recognizing the cloud droplets (25), where an input of the trained neural network is the at least one hologram and where an output of the trained neural network comprises particle size information of the cloud droplets (25), and- calculating (208), in the processor, a presence of possible icing conditions based on the obtained ambient temperature information of the UAV and the particle size information of the cloud droplets (25).

2. The method according to claim 1, characterized in that preprocessing the hologram (204) by performing a background removal for the hologram, the background comprising fixed objects shown in the diffraction image.

3. The method according to claim 1, characterized in that preprocessing the hologram (204) by changing a size of the diffraction image.

4. The method according to claim 1, characterized in that providing information from the UAV, comprising information on whether propellers are turned on or off, to the processor.

5. The method according to claim 1, characterized in that providing motor power and / or rotation speed of propellers and / or size of the propellers from the UAV to the processor.

6. The method according to claim 1, characterized in that providing air speed of the UAV to the processor.

7. The method according to claim 1, characterized in that GPS data of the UAV is provided to the processor in order to localize the obtained information, and for allowing the distribution of the obtained information also to other devices, servers and / or to other interested users and / or parties and / or to other flying unmanned or manned vehicles or devices.

8. The method according to claim 7, characterized in that the localized information on possible icing conditions are applied in assisting in making a localized weather forecast.

9. The method according to claim 1, characterized in that if the icing conditions are detected and calculated to be present, also the magnitude of the icing conditions are determined.

10. The method according to claim 1, characterized in that determining, from the calculated presence of possible icing conditions, and possibly also from its magnitude, an indication on whether it is safe to fly the UAV from such an inspection moment onwards.

11. The method according to claim 10, characterized in that showing a safety parameter of the flying conditions for the UAV as a numeric value or as a traffic lights style indicator to a user or a manager of the UAV.

12. The method according to claim 1, characterized in that the trained neural network is configured to recognize liquid water droplets and solid ice crystals in the hologram and to separate liquid and solid particles in the analysis.

13. The method according to claim 1, characterized in that the method comprises the step of:starting the recording step only after the sensed temperature reaches a value below a threshold value, where the threshold value has been set within the temperature range of +2 … +5 °C.

14. The method according to claim 1, characterized in that the image detector (22) is placed in a closed volume where the closed volume is restricted by a window or by double windows locating opposite of one another,where a heating element is placed within the closed volume for managing the temperature within the closed volume and on the surfaces of the window(s).

15. The method according to claim 14, characterized in that the operation and the switching on / off the heating element is based on the measured temperature (206), wherein further, the heating element is switched on when liquid droplets have been detected and the measured temperature (206) locates between the temperature range of 0 … +2 °C.

16. The method according to claim 1, characterized in that the ambient temperature information of the UAV (206) is obtained by sensing temperature by a temperature sensor comprised within the UAV or as fixed to the UAV.

17. The method according to claim 1, characterized in that the ambient temperature information of the UAV (206) is obtained by using an infrared camera fixed to the UAV.

18. The method according to claim 1, characterized in that the ambient temperature information of the UAV (206) is obtained by receiving data from an external data source or server comprising a local weather forecast.

19. The method according to claim 1, characterized in that the method further comprises the steps of:summing information of at least two consecutive diffraction images (25a) for increasing a number of liquid droplets within a single, summed, image; andusing the summed image in the calculations performed by the processor, noting the number of consecutive summed images in the calculations as well.

20. An arrangement for detecting presence of possible icing conditions of an unmanned aerial vehicle, UAV, characterized in that the arrangement comprises:– temperature information collection means configured to obtain (206) ambient temperature information of the UAV,– an image detector (22) of a digital in-line holography apparatus (1) configured to record (201) at least one diffraction image (25a) comprising information of cloud droplets (25) situated in an open measurement space, – a computer (3), where the at least one recorded diffraction image (25a) is / are transferred (202) to as respective holograms, wherein the computer (3) comprises a processor,– the processor configured to apply (205) a trained neural network for recognizing the cloud droplets (25), where an input of the trained neural network is the at least one hologram and where an output of the trained neural network comprises particle size information of the cloud droplets (25), and – the processor configured to calculate (208) a presence of possible icing conditions based on the obtained ambient temperature information of the UAV and the particle size information of the cloud droplets (25).

Images