Method for detecting aircraft icing and device and aircraft therefor

Abstract

The invention relates to a method for detecting aircraft icing of an aircraft, the method comprising the following steps: - Recording at least a part of an outer flow surface of the aircraft by means of at least one optical image sensor of a detection device and generating digital image data containing the recorded part of the outer flow surface of the aircraft, - inputting the digital image data into a machine learning system of the detection device as input data, which has learned a correlation between digital image data as input data and various types of aircraft icing as output data by means of at least one machine-learned decision algorithm in order to obtain output data, and - Detecting aircraft icing depending on the output data received and, if aircraft icing has been detected, classifying the aircraft icing with regard to at least one icing type depending on the output data received.

Description

[0001] The invention relates to a method for detecting icing of an aircraft, particularly during flight operations. The invention also relates to a detection device and an aircraft for this purpose.

[0002] Aircraft have aerodynamic surfaces that are subjected to a flow of air flowing at a certain speed, generating a lift force that allows the aircraft to fly within the atmospheric air masses. However, depending on the flight situation, such aerodynamic surfaces can be susceptible to icing, resulting in a layer of ice (partial or complete) forming on the outer surface of the flow, which is surrounded by the surrounding air masses. This layer can alter the aerodynamic properties of the aerodynamic surface and thus negatively impact the overall flight condition. Icing on the fuselage or other surfaces that are not primarily required to generate lift can also negatively impact the aircraft's flight characteristics, which can quickly lead to critical situations, especially during takeoff and landing.

[0003] Detecting aircraft icing, i.e., the partial or complete formation of ice on the outer airfoil surfaces of an aircraft, especially during flight, is not trivial. Manual inspection of the aerodynamic surfaces, for example, for icing, is not possible in flight because access to the aerodynamic surfaces of the aircraft is precluded for the flight crew. Other systems that rely on sensory detection of icing are sometimes highly complex and technically vulnerable, which ultimately leads to a high false alarm rate and thus low acceptance by pilots.

[0004] US Pat. No. 6,253,126 B1 discloses a method and device for flight monitoring, wherein a series of additional air pressure sensors are arranged on the aircraft, particularly on the wings, in order to determine important flight parameters by monitoring the wing profile pressure as continuously as possible. These sensors are also intended to be able to detect icing.

[0005] The disadvantage here, however, is that no holistic concept is provided, so that icing, for example, is not detected between the sensors, or only partial icing on one of the sensors is detected as icing. In the first case, icing is not detected, which can have an overall negative impact on flight characteristics and thus increase the potential risk of accidents. In the second case, icing is detected, whereupon countermeasures may be taken by the pilot, even though the icing does not represent a safety-relevant impairment of flight characteristics. In such a case, the countermeasures taken, such as changing the flight path, would lead to higher costs and longer flight times, even though this was not necessary. Furthermore, the method proposed in US 6 253 126 B1 is very complex to develop, install and maintain.In addition, such a device would be very heavy (equipment, power supply, data communication to the computing unit that is supposed to perform the non-trivial evaluation), which would most likely lead to increased fuel consumption.

[0006] US Pat. No. 8,692,361 B2 discloses a method for monitoring the flow quality of aircraft aerodynamic surfaces, where the main characteristic of the monitored flow is laminarity. To achieve this, the aerodynamic surface is heated, forcing an early transition from laminar to turbulent flow. Based on drag data recorded during laminar and turbulent flow, fouling of the aerodynamic surface can now be detected.

[0007] The disadvantage here is that both a complex heating system and a complex sensor system are necessary to detect the corresponding negative influences on the aerodynamic surface. Icing or contamination of the aerodynamic surfaces that could impair the aircraft's characteristics cannot be reliably detected this way, as this method only allows for an analyzable measurement of the aerodynamic quality of the surface between a "perfectly clean" condition and a "slightly dirty" or "slightly icy" condition. If the aerodynamic quality of the surface deteriorates further, no difference can be determined from the method described in US 8,692,361 B2. The degradation, which could potentially become safety-critical (e.g., in the case of severe icing), lies beyond the range for which the described method can be used.

[0008] US Pat. No. 6,304,194 B1 discloses a method and device for detecting icing of an aircraft in flight, determining aircraft performance values. A sensor model is used that maps the sensor values ​​in the non-degraded state. The sensor values ​​derived from the sensor model are then compared with the measured sensor values.

[0009] US 2016 / 0 035 203 A1 and US 2014 / 0 090 456 A1 each disclose the icing of an aircraft based on turbine power.

[0010] DE 10 2016 111 902 A1 and WO 2018 / 002 148 A1 disclose a method and a device for detecting aircraft icing in flight. Current flight status data of the aircraft in flight are first determined. Based on this, a flight performance indicator is then calculated. A flight performance model is then used to determine a nominal flight performance reference indicator. By comparing both indicators, aircraft icing is then determined by detecting a degradation in flight performance. The disadvantage of this method is that aircraft icing is only detected once the aircraft icing has already led to a degradation in flight performance.

[0011] Optical detection methods are known from CN 1 14 596 315 A, CN 1 12 966 692 A and CN 1 14 372 960 A, in which the aerodynamic profile surfaces of the aircraft are recorded with the help of a camera and these image data are then fed into an AI system that has learned the corresponding aircraft icing from the image data.

[0012] CN 1 17 994 203 A discloses image-based icing detection, whereby corresponding image data is input into a trained neural network, and the determination of whether or not icing is obtained as output data. However, this not only provides image-based detection, but also detection by means of vibration of the outer skin by a probe.

[0013] US 10 160 550 B1 also discloses the detection of icing using an artificial neural network, whereby in particular so-called image features such as color, texture and edges were learned in order to then be able to recognize them as icing in the images.

[0014] US 2023 / 0 219 698 A1 discloses the monitoring of the aircraft's outer skin using an optical detection system that records image data of the outer skin and transmits it to the crew. In addition, it is planned to detect such icing from the image data using image recognition based on an artificial neural network.

[0015] It is therefore an object of the present invention to provide an improved method and an improved device for detecting aircraft icing, in particular of an aircraft in flight.

[0016] The object is achieved according to the invention with the method according to claim 1. Advantageous embodiments of the invention can then be found in the corresponding subclaims.

[0017] According to claim 1, a method for detecting aircraft icing of an aircraft is proposed, the method according to the invention comprising the following steps: - Recording at least a part of an outer flow surface of the aircraft by means of at least one optical image sensor of a detection device and generating digital image data containing the recorded part of the outer flow surface of the aircraft, - inputting the digital image data into a machine learning system of the detection device as input data, which has learned a correlation between digital image data as input data and various types of aircraft icing as output data by means of at least one machine-learned decision algorithm in order to obtain output data, and - Detecting aircraft icing depending on the output data received and, if aircraft icing has been detected, classifying the aircraft icing with regard to at least one icing type depending on the output data received.

[0018] All surfaces that face external ambient air and / or are flowed around by external ambient air of the aircraft are referred to as external flow surfaces in the context of the present invention.

[0019] An aircraft within the meaning of the present invention refers to both manned aircraft and unmanned aircraft (drones). The term "aircraft" encompasses both fixed-wing aircraft (aircraft in general, in particular commercial aircraft as a means of mass transit and transport aircraft; but also recreational aircraft and jets) and rotary-wing aircraft, in particular helicopters.

[0020] Accordingly, the invention provides for a machine learning system comprising at least one machine-trained decision algorithm. This can be, for example, an artificial neural network. The at least one machine-trained decision algorithm has learned at least one correlation between digital image data as input data and various types of aircraft icing as output data. These digital image data, which are input to the decision algorithm, comprise images of the outer flow surface of an aircraft, for example the leading edge of wings. The decision algorithm has been trained to the effect that not only aircraft icing can be detected from the digital image data, but also which type of aircraft icing is involved.Accordingly, the detected aircraft icing is classified according to an icing type by the machine-trained decision algorithm.

[0021] Particularly during flight, but also on the ground, the desired outer flow surface is recorded using at least one optical image sensor. Several optical image sensors can be provided, each of which detects partial areas of an outer flow surface. An optical image sensor therefore records at least a partial area of ​​an aircraft's outer flow surface, for example, part of the leading edge of a wing, and generates digital image data from the recordings, which are then fed to the machine learning system. The recording by the optical image sensor and the generation of the digital image data take place continuously, so that a digital image of the recorded partial area of ​​the outer flow surface is created at discrete time intervals.For example, it is conceivable that a digital image is generated at intervals of 1 second and then fed into the machine learning system.

[0022] The machine learning system provides these digital image data, which comprise at least a single image of the recorded partial area and can advantageously also contain several images that were recorded one after the other, to the machine-trained decision algorithm as input data, whereupon the decision algorithm provides corresponding output data.

[0023] This output data contains not only information about whether aircraft icing can be detected in the digital image data of the input data, but also the type of icing. Only from the output data can both the detection of aircraft icing and the type of icing be directly or indirectly derived and presented to the pilot or other monitoring authorities so that appropriate countermeasures can be initiated.

[0024] With the help of the present invention, it is thus possible to detect aircraft icing and classify the type of icing without installing sensors in the outer airfoil, regardless of the aircraft in question. This allows compliance with current design regulations for new aircraft, which require the precise detection of ice deposits. Furthermore, the maintenance and acquisition costs for such a detection device can be significantly reduced, since, apart from optical sensors and a digital processing unit running the machine learning system, no additional hardware is required.

[0025] It can be provided that each recorded image is entered into the decision algorithm separately, independently of the previously recorded image data, so that a separate decision is made for each recorded image, independent of the previous images.

[0026] However, the inventive concept also encompasses the use of one or more previously made decisions by the decision algorithm as additional input data for the current mapping, so that a temporal progression of the machine learning system's decisions is also mapped. The results of the last x-mappings are thus included in the result calculation of the most recent mapping (x+1) and are evaluated. This results in greater accuracy and fewer "false positive" decisions. For example, the last 15 seconds of each mapping can also be considered, whereby the number of x-mappings considered can also be varied depending on the probabilities of the classifications.

[0027] Accordingly, in this embodiment, the output data and / or information derived therefrom from previous iteration steps are additionally input into the at least one machine-trained decision algorithm as input data in the subsequent iteration step. Furthermore, such output data or information derived therefrom can, in principle, relate not only to the classification itself, but also to the probability with which this classification was made by the decision algorithm.

[0028] According to one embodiment, the icing types are clear ice, frost ice, mixed ice, droplet ice (SLD ice), return ice, and / or inter-cycle ice. Droplet ice is ice formed by large, supercooled droplets.

[0029] To learn the decision algorithm, various images of these icing types as well as images of uniced surfaces are used to train the decision algorithm to classify them automatically.

[0030] According to one embodiment, the method is further developed as follows: - Recording at least the part of the outer flow surface of the aircraft by means of at least the optical image sensor of the detection device several times and in succession and generating several digital image data in succession which contain a temporal profile of the recorded part of the outer flow surface of the aircraft, - Inputting the multiple, temporally consecutive digital image data into the machine learning system of the detection device as input data, which has learned a correlation between multiple, temporally consecutive digital image data as input data and various change states of aircraft icing as output data by means of at least one machine-learned decision algorithm in order to obtain output data, - Detecting a temporal progression of aircraft icing changes depending on the received output data.

[0031] Accordingly, the invention provides a machine learning system with a machine-trained decision algorithm that has learned a correlation between multiple, chronologically consecutive digital image data as input data and various change states of aircraft icing as output data. Such change states can, for example, be an areal increase in icing, a areal decrease in icing, and / or an areal unchange in icing. The multiple digital image data, which were recorded and generated chronologically consecutively, thus show the recorded portion of the outer flow surface over a certain period of time in the past and thus contain a history of the change in aircraft icing.

[0032] It is conceivable that, to obtain the output data, the majority of the consecutive digital image data are provided to the decision algorithm as input data each time, so that a certain number of the acquired digital image data from the past are temporarily stored for this purpose. It is also conceivable, however, that the decision algorithm itself temporarily stores the icing condition from the past and, when inputting current digital image data, then infers a change based on the temporary storage of the previous icing conditions, so that a progression of the icing can be detected.

[0033] This decision algorithm can be the same decision algorithm according to claim 1. However, it is also conceivable that it is an additional, further decision algorithm that provides a result in addition to the first decision algorithm according to claim 1. The results from both decision algorithms can then be processed, for example, using another AI, such as a support vector machine (SVM or SVN - Support Vector Network), to obtain a corresponding final result.

[0034] According to one embodiment, it is provided that the digital image data are input into the machine learning system of the detection device as input data, wherein a correlation between digital image data as input data and an icing area of ​​the aircraft icing has been learned by means of at least one machine-learned decision algorithm, wherein a measure of the icing area of ​​the aircraft icing is determined as a function of the output data obtained.

[0035] This can also be the same decision algorithm according to claim 1. However, it is also conceivable that it is an additional, further decision algorithm that provides a result in addition to the first decision algorithm according to claim 1. The results from all decision algorithms can then be processed, for example, using another AI, such as a support vector machine (SVM or SVN - Support Vector Network), to obtain a corresponding final result.

[0036] According to this embodiment, by providing the digital image data as input data to the decision algorithm, a measure of the icing area of ​​the aircraft can be determined in order to determine the icy area. The measure of the icing area can be specified as a percentage or on an alternative scale.

[0037] According to one embodiment, it is provided that the digital image data are input into the machine learning system of the detection device as input data, wherein by means of at least one machine-learned decision algorithm a correlation between digital image data as input data and de-icing measures on an outer flow surface of an aircraft has been learned, wherein it is determined whether de-icing measures are / were carried out or not depending on the output data obtained.

[0038] This makes it possible to determine whether de-icing measures are / were being carried out or not. If aircraft icing has been detected and the pilot or an automatic system has taken appropriate de-icing measures, this can be detected by the present method. Furthermore, if the temporal change in icing is also determined, as already described in one embodiment above, it can be determined in this context whether the de-icing measures implemented have led to the desired result (reduction in the de-icing area) or not.

[0039] In this embodiment, the optical image sensor is directed toward the outer flow surface in such a way that it records the de-icing measures that are visible on the outer flow surface when activated. Upon activation, these de-icing measures are also included in the image data. Such de-icing measures can be, for example, "deflating boots."

[0040] This can also be the same decision algorithm according to claim 1. However, it is also conceivable that it is an additional, further decision algorithm that provides a result in addition to the first decision algorithm according to claim 1. The results from all decision algorithms can then be processed, for example, using another AI, such as a support vector machine (SVM or SVN - Support Vector Network), to obtain a corresponding final result.

[0041] In principle, the results of the decision algorithm(s) can be incorporated into a validation process, with which the trained decision algorithm in question is checked and further trained.

[0042] According to one embodiment, it is provided that the output data obtained from the decision algorithm and / or the information derived therefrom are displayed on a display device.

[0043] This could, for example, be a display in the cockpit to indicate to the pilot that there is icing and, if applicable, the type of icing.

[0044] The object is also achieved with the detection device according to claim 7, which is configured to carry out the method described above. For this purpose, the detection device has at least one imaging sensor connected to a computing unit on which the machine learning system with the machine-trained decision algorithm is executed.

[0045] The object is further also achieved according to the invention with an aircraft having such a detection device.

[0046] The invention is explained by way of example with reference to the accompanying figures. They show: Fig. 1: schematic representation of the detection device for carrying out the method according to the invention; Fig. 2: Illustration of a training of the machine-learned decision algorithm in the context of the decision-making process.

[0047] Fig. Figure 1 shows, in a highly simplified schematic representation, a wing 10 having a leading edge 11. During flight, icing 12 has formed on the leading edge 11, which has spread extensively over the entire area of ​​the leading edge 11. The present invention is intended to detect this aircraft icing 12. The Fig. The icing shown in Figure 1 is directed only at the leading edge of the wing as an example, whereby the present method can detect icing regardless of the position on the aircraft.

[0048] A detection device 100 is located on or in the aircraft (not shown), which has at least one camera 110 with at least one digital image sensor 112. The camera 110 with the digital image sensor 112 is directed at the wing 10, in particular at the wing leading edge 11, and can be arranged, for example, behind a glass pane on the fuselage of the aircraft.

[0049] The digital image sensor 112 now records the leading edge 11 of the wing 10 and generates digital image data from the recording, which is then forwarded to a computing unit 120. A machine learning system 122 with a machine-trained decision algorithm is executed on the computing unit 120.

[0050] The machine learning system 122 is provided in such a way that, with its trained decision algorithm, it has learned at least one correlation between digital image data as input data and various types of aircraft icing as output data. Such icing types can be, for example, clear ice, rime ice, mixed ice, droplet ice (SLD), return ice, and / or inter-cycle ice.

[0051] The digital image data originating from the digital image sensor 112 is then provided as input data to the machine learning system 122. The output data obtained is whether aircraft icing is present and, if so, which type of icing can be deduced from the digital image data. The result of this determination can then be displayed on a display 130.

[0052] The digital image data originating from the digital image sensor 122 can also be temporarily stored in a digital data storage 140 so that they can also be entered into the machine learning system 122 as input data at a later time in order to detect a temporal progression of aircraft icing. It is conceivable that each image stored in the digital data storage 140 is associated with a previously detected type of icing and a determined measure of the icy area, so that a change in aircraft icing can also be detected via this.

[0053] Fig.Figure 2 provides an overview of the various stages for both training the decision algorithm and classifying the icing type. Three different phases can be identified: a first training phase, a second test phase, and a third evaluation phase.

[0054] The test phase and the evaluation phase can be carried out both during the creation of the trained decision algorithm and in real-time operation.

[0055] During the initial testing phase, the images are assessed for quality and content. Depending on the camera position, other characteristics arise that must be evaluated. This requires data processing, including classification, scaling, resizing, etc. In particular, resolution, grayscale, chromaticity, color characteristics, and value are normalized.

[0056] Depending on availability, flight-specific information such as current weather conditions, position, altitude, and humidity can also be used to provide a more consistent display. This also applies particularly to other sensors. This information is not mandatory, but it does enable more accurate and better training of the data sets.

[0057] Noisy images are already excluded at this point because they do not meet the required quality / features. Using support vector machines classification, an n-dimensional hyperplane is then determined, which provides a distinction between the individual objects. In cases where no hyperplanes can be determined directly due to low chromaticity, a distinction is determined using n-dimensional support vector regressions.

[0058] The resulting model is used after an optional validation in active flight operations (test phase) or for a comprehensive analysis (evaluation).

[0059] In the test phase, an image is fed into the decision algorithm, classified according to its quality (preprocessing), and then processed. For this purpose, resolution, grayscale, chromaticity, color characteristics, and value are normalized. Postprocessing provides several stages that determine the icing area and whether any de-icing measures, such as de-icing boots, are active.

[0060] The results are stored sequentially, enabling a holistic analysis. The evaluation utilizes the temporal progression of icing and can, even with short loops, test the functionality of de-icing mechanisms such as de-icing boots and provide short-term forecasts. A key aspect here is error reduction. A high frame rate prevents icing changes from occurring suddenly. Viewing an image sequence (n previous images, random start images, and large time jumps) sharpens and strengthens the accuracy of the hyperplanes, preventing any misinterpretations, such as those that occur with high deltas.

[0061] After successful training and validation, the method can be transferred to other aircraft regardless of the need for further training, so that ultimately only the test phase with the generated model can be used. To ensure reliable results, the output of previous calculations is also used as input for new image interpretations. List of reference symbols 10 wings 11 Wing leading edge 12 Aircraft icing 100 detection device 110 Camera 112 imaging sensor 120 computing units 122 machine learning system 130 displays 140 digital data storage

Claims

[1] Method for detecting aircraft icing (12) of an aircraft, wherein the method comprises the following steps: - Recording at least a part of an outer flow surface of the aircraft using at least one optical image sensor of a detection device (100) and generating digital image data which includes the recorded part of the outer flow surface of the aircraft, - Inputting the digital image data into a machine learning system (122) of the detection device (100) as input data, which has learned a correlation between digital image data as input data and different types of icing of an aircraft (12) as output data by means of at least one machine-trained decision algorithm, in order to obtain output data, and - Detecting aircraft icing (12) depending on the output data received and, if aircraft icing (12) has been detected, classifying the aircraft icing (12) with respect to at least one type of icing depending on the output data received. [2] Method according to claim 1, characterized by that the types of icing are clear ice, frost ice, mixed ice, droplet ice (SLD), return ice and / or inter-cycle ice. [3] Method according to claim 1 or 2, characterized by - Recording at least part of the outer flow surface of the aircraft using at least the optical image sensor of the detection device (100) multiple times and successively, and generating several successive digital image data sets containing a temporal progression of the recorded part of the outer flow surface of the aircraft, - Inputting the multiple, temporally successive digital image data into the machine learning system (122) of the detection device (100) as input data, which has learned a correlation between multiple, temporally successive digital image data as input data and various change states of an aircraft icing (12) as output data by means of at least one machine-trained decision algorithm, in order to obtain output data, - Detecting a temporal progression of the change in aircraft icing (12) depending on the output data obtained. [4] Method according to any one of the preceding claims, characterized by, that the digital image data are entered into the machine learning system (122) of the detection device (100) as input data, wherein a correlation between digital image data as input data and an icing area of ​​the aircraft icing (12) has been learned by means of at least one machine-trained decision algorithm, wherein a measure for the icing area of ​​the aircraft icing (12) is determined as a function of the output data obtained. [5] Method according to any one of the preceding claims, characterized by, that the digital image data are entered into the machine learning system (122) of the detection device (100) as input data, wherein a correlation between digital image data as input data and de-icing measures on an external flow surface of an aircraft has been learned by means of at least one machine-trained decision algorithm, wherein it is determined, depending on the output data obtained, whether de-icing measures are carried out or not. [6] Method according to any one of the preceding claims, characterized by that the output data obtained from the decision algorithm and / or the information derived therefrom are displayed on a display device. [7] Detection device (100) for detecting aircraft icing (12) of an aircraft configured to carry out the method of one of the preceding claims. [8] Aircraft with a detection device (100) according to claim 7.

Images