Optical aircraft monitoring using reflective external surfaces

Abstract

Method for optically monitoring an area (15) of an aircraft (1), - wherein images of the area to be monitored (15) are taken in flight of the aircraft (1) with a camera (9) fixed to the fuselage (3) of the aircraft (1) and - and wherein the images are analyzed, characterized in that a field of view of the camera (9) with a reflective outer surface (13) of the aircraft (1) is deflected at least partially towards the area (15) to be monitored.

Description

TECHNICAL FIELD OF THE INVENTION

[0001] The invention relates to a method for optically monitoring an area of ​​an aircraft. In particular, the invention relates to a method having the features of the preamble of independent patent claim 1. Furthermore, the invention relates to an aircraft having a fuselage, an area to be monitored, and a camera fixedly mounted on the fuselage of the aircraft, the field of view of which is directed at least partially toward the area to be monitored.

[0002] The area to be monitored can be a wing or half-wing. Optical monitoring can be performed, for example, to detect deformations of wings during aircraft flight. STATE OF THE ART

[0003] EP 2 772 439 B1 discloses a method for identifying a position of a surface on an aircraft, in which method image data for an image of the surface on the aircraft is received, in which method the image data is processed to determine whether the position of the surface on the aircraft is a desired position, and in which method a surface position identification report is generated which includes information which identifies whether the position of the surface on the aircraft is the desired position. The image of the surface is obtained when the aircraft is in flight. The image data is received by a camera on the aircraft. Specifically, two cameras are arranged on a horizontal stabilizer of the aircraft, each of whose fields of view includes one half of the aircraft's wing.

[0004] For ultralight aircraft, such as unmanned stratospheric aircraft, which are designed with extremely lightweight construction, arranging cameras on the horizontal stabilizer, i.e., at a great distance from the aircraft's center of mass, proves complex. Firstly, additional support structures are required. Secondly, the cameras, located at a great distance from the center of mass, require the installation of counterweights. Since the counterweights at the front of the fuselage cannot be positioned at the same distance from the center of mass as the vertical stabilizer, they must be significantly larger than the mass of the camera to ensure a trimmed flight condition. This poses the risk of exceeding the maximum take-off mass.

[0005] EP 0 715 743 B1 discloses a panoramic imaging system for a room, comprising a camera and a mirror. The mirror is a dome-shaped convex mirror with a curved surface. The camera is mounted relative to the mirror such that at least a major portion of the mirror's surface is within the camera's field of view. The camera is arranged directly below the mirror, in particular in a hollow hemisphere, the reflective surface of which forms an additional mirror opposite the dome-shaped mirror. The camera records a distorted image of the room across the additional mirror and the dome-shaped mirror. A signal processing device is programmed to generate a distorted image of the room. Furthermore, the signal processing device is designed to detect and display movements within the room.The well-known panoramic image system can be used as airport surveillance for panoramic monitoring of an airfield from a control tower or on the belly of aircraft to enable pilots to view obscured areas below the passenger compartment of an aircraft.

[0006] US 2008 / 0 185 526 A1 discloses a device and method for aligning the field of view of a stationary camera. Movable mirrors are used to direct an image from a desired direction onto the camera lens. Specifically, the mirrors are designed to be pivoted into or out of an active position to align the camera's field of view in various, mutually orthogonal or opposite directions.

[0007] From DK Bilal, et al.: “Realtime localization and estimation of load on aircraft wings from depth images”, sensors 2020, special issue “Optical sensors for structural health monitoring”, it is known that depth sensors arranged on the landing gear of an aircraft can be used to detect deformations of the aircraft’s half wings.

[0008] DE 10 2006 031 009 A1 discloses a method for monitoring the condition of structural components of an aircraft. In this method, an optical sensor, in conjunction with a processing unit, determines image deviations and, consequently, changes in the shape of the structure from successive images of the structural components to be monitored. To generate the sharpest and highest-resolution images possible, the sensor is continuously aligned with various smaller sub-areas of the structure to be monitored. This is achieved by expanding the viewing angle of a stationary camera using a drivable deflecting mirror. The camera's lens is fixedly mounted on a structure and faces a mirror. The mirror is driven and pivots around one or more axes.

[0009] US 2021 / 0 295 723 A1 discloses a method for calculating the locations or areas of objects in an area outside an aircraft. Pulses of polarized light are emitted from the aircraft, and a portion of the projected pulses of polarized light that is reflected back by an object in the monitored area and is polarized orthogonally to the projected light pulses is recorded by a camera through polarizing filters to capture a two-dimensional image of the monitored area. OBJECT OF THE INVENTION

[0010] The invention is based on the object of providing a method for optically monitoring an area of ​​an aircraft having the features of the preamble of patent claim 1 and an aircraft having the features of the preamble of patent claim 6, which can be implemented without problems with regard to the maximum take-off mass in or as an ultralight aircraft and / or stratospheric aircraft. SOLUTION

[0011] The object of the invention is achieved by a method having the features of independent claim 1 and an aircraft having the features of independent claim 6. Dependent claims 2 to 5 relate to preferred embodiments of the method according to the invention. The further dependent claims define both preferred embodiments of the method according to the invention and of the aircraft according to the invention. DESCRIPTION OF THE INVENTION

[0012] In a method for optically monitoring an area of ​​an aircraft, in which images are recorded with a camera fixedly mounted on the fuselage of the aircraft and the images are analyzed, a field of view of the camera is, according to the invention, at least partially redirected towards the area to be monitored by means of a reflective outer surface of the aircraft.

[0013] The outer surface of the aircraft is part of the outer skin of the aircraft, over which air flows during flight. In the method according to the invention, such an outer surface, which is sufficiently reflective or is made reflective, is used to redirect the field of view of the camera, which is fixedly mounted on the fuselage of the aircraft, to the area of ​​the aircraft to be monitored. This allows the camera to be arranged on the fuselage of the aircraft where the installation of the camera is as inexpensive as possible. Suitable outer surfaces for this purpose are already present on many aircraft, or they can be created by simply shaping the outer shell of the aircraft without impairing its functionality.

[0014] An exterior surface of the aircraft suitable for implementing the method according to the invention will often be curved. This leads to distortion of the camera images captured across the reflective exterior surface. Therefore, it is preferable to calculate out the distortion of the images of the aircraft area to be monitored, which is due to reflection from the reflective exterior surface, when analyzing the images. The information required for this can be acquired empirically by arranging known geometric structures in the area to be monitored and comparing them with the corresponding camera images.

[0015] Not only in extremely lightweight aircraft, deformations will occur between the location of the fuselage where the camera is mounted and the reflective outer surface. These deformations are best taken into account when calculating distortion in images of the area being monitored. The positions of optical markers on the reflective outer surface can be determined in the images recorded by the camera and taken into account when calculating distortion. Specifically, the position of the optical markers can be used to determine the position of the reflective outer surface relative to the camera, which can then be taken into account when calculating distortion.

[0016] In addition, the position of the camera can be identified in the camera images to detect deformations between the location of the hull where the camera is mounted and the reflective outer surface and to take them into account when calculating the distortion of the images of the area to be monitored.

[0017] When analyzing the images, the positions of optical markers in the monitored area can also be identified in the camera images. This can make it easier to detect, for example, deformation of the monitored area through image processing.

[0018] Furthermore, camera movements can be detected and taken into account when evaluating the camera images. For example, digital image stabilization can be implemented.

[0019] In an aircraft having a fuselage, an area to be monitored and a camera fixedly arranged on the fuselage of the aircraft, the field of view of which is directed at least partially towards the area to be monitored, according to the invention a reflective outer surface of the aircraft redirects the field of view of the camera at least partially towards the area to be monitored.

[0020] In many cases, the reflective outer surface of the inventive method and the inventive aircraft will have a convex curve. The targeted curvature can be specifically selected and used to expand the camera's field of view so that it covers the entire area of ​​the aircraft to be monitored.

[0021] Specifically, the reflective outer surface can be a surface of the aircraft's vertical and / or horizontal stabilizer. If the camera is positioned near the aircraft's center of mass, i.e., at a distance in front of the reflective outer surface, the camera's field of view can be expanded by the reflective outer surface to cover the entire wing of the aircraft.

[0022] In the method and the aircraft according to the invention, the camera is located at a relatively large distance of at least 1 m, preferably at least 2 m, and even more preferably at least 3 m from the reflective outer surface. The camera's original field of view can therefore be comparatively small, since it only needs to cover the reflective outer surface. From the reflective outer surface, it is then expanded to the area to be monitored.

[0023] The area of ​​the centre of gravity is understood in particular to mean an area in the direction of the longitudinal extent of the aircraft which is not more than 10% and preferably not more than 5% of the maximum longitudinal extent of the aircraft from the centre of gravity of the aircraft.

[0024] As already indicated, the area of ​​the aircraft to be monitored with a camera can be an entire wing of the aircraft. However, it can also be half of the aircraft's wing, or just a part of it, such as the outer area of ​​a half-wing of the aircraft.

[0025] As already explained in connection with the method according to the invention, optical markers can also be attached to the reflective outer surface of the aircraft according to the invention or in the area to be monitored.

[0026] The optical markers can be striped or dot markings. They can also include light sources. It goes without saying that the wavelength of the light sources must be matched to the sensitivity of the camera. The camera can be tuned to wavelengths in the visible and / or invisible range.

[0027] An illumination light source can be assigned to the camera. Specifically, an illumination light source aligned coaxially with the camera can be arranged on the camera. The wavelength of the illumination light must also be adapted to the wavelength range in which the camera is light-sensitive.

[0028] Accelerometers can be attached to the camera to detect camera movements.

[0029] The reflective outer surface can be polished and / or painted. Generally, it is not necessary to apply a complex metal coating or similar to the outer surface.

[0030] The aircraft may in particular be an ultralight aircraft and / or a stratospheric aircraft, i.e. an aircraft constructed in an extremely lightweight manner, on which it is not readily possible to install cameras outside the area of ​​the centre of gravity of the respective aircraft without extensive additional measures.

[0031] Advantageous further developments of the invention emerge from the patent claims, the description and the drawings.

[0032] The advantages of features and combinations of several features mentioned in the description are merely exemplary and can be effective alternatively or cumulatively, without the advantages necessarily having to be achieved by embodiments according to the invention.

[0033] With regard to the disclosure content – ​​not the scope of protection – of the original application documents and the patent, the following applies: Further features can be found in the drawings – in particular the illustrated geometries and the relative dimensions of several components to one another, as well as their relative arrangement and operative connection. The combination of features of different embodiments of the invention or features of different patent claims is also possible, deviating from the chosen references of the patent claims, and is hereby encouraged. This also applies to features that are illustrated in separate drawings or mentioned in their description. These features can also be combined with features of different patent claims.Likewise, features listed in the patent claims may be omitted for further embodiments of the invention, but this does not apply to the independent patent claims of the granted patent.

[0034] The number of features mentioned in the patent claims and the description is to be understood as meaning that exactly this number or a greater number than the stated number is present, without the need for the explicit use of the adverb "at least." Thus, for example, if an acceleration sensor is mentioned, this is to be understood as meaning that exactly one acceleration sensor, two acceleration sensors, or more acceleration sensors are present. The features mentioned in the patent claims may be supplemented by further features or may be the only features present in the subject matter of the respective patent claim.

[0035] The reference signs contained in the patent claims do not represent a limitation of the scope of the subject-matter protected by the patent claims. They serve solely to make the patent claims easier to understand. BRIEF DESCRIPTION OF THE CHARACTERS

[0036] In the following, the invention is further explained and described with reference to preferred embodiments shown in the figures. Fig. 1 shows a stratospheric aircraft with a reflective outer surface on its vertical stabilizer, using which the method according to the invention is carried out, as an example of an aircraft according to the invention; and Fig. 2 shows another stratospheric aircraft with a reflective outer surface on its horizontal stabilizer, using which the method according to the invention is carried out, as a further example of an aircraft according to the invention. FIGURE DESCRIPTION

[0037] The Fig. The aircraft 1 shown in Figure 1 is an extremely lightweight, so-called stratospheric aircraft 2, which is configured for unmanned flight at very high altitudes. The aircraft 1 has a fuselage 3, a wing 4 with two half-wings 5 ​​attached to the fuselage 3, a horizontal stabilizer 6, and a vertical stabilizer 7, which are arranged at the rear end of a tail boom 8 of the fuselage 3. A camera 9 is fixedly attached to the fuselage 3 in the area of ​​the center of mass 16 of the aircraft 1. Due to the proximity of the camera 9 to the center of mass 16, trim masses to compensate for the camera 9 can be dispensed with, or small trim masses are sufficient. The original field of view 10 of the camera 9 has only a small aperture angle 11 and is directed towards a leading edge 12 of the vertical stabilizer 7.The leading edge 12 has a convex, mirrored outer surface 13, which expands the original field of view 10 into a horizontally expanded field of view 14 with a larger aperture angle 17, covering the entire wing 4 and also the camera 9 itself. Thus, the entire wing 4 with both half-wings 5 ​​is depicted as the area 15 to be monitored in the images from camera 9. Due to the convex curvature of the mirrored outer surface 13, these images are distorted. However, this distortion can be calculated out of the images from camera 9. In . Fig. 1 shows optical markers 18 in the area 15 to be monitored, the positions of which can be particularly easily determined in the images from camera 9, for example, to detect deformations of the wing 4. Such an optical marker can also be arranged on the reflective outer surface 13, or the edge of the reflective outer surface can serve as such an optical marker, the position of which is determined in the images from camera 9 in order to detect the relative position of the reflective outer surface 13. This relative position affects the distortion of the area 15 to be monitored in the images from camera 9, and this relative position can vary, for example, due to deformation of the tail boom 8. The relative position of the reflective outer surface 13 to camera 9 can also be detected based on the position of camera 9 in the images recorded with camera 9.

[0038] Also in Fig. The aircraft 1 shown in Figure 2 is a so-called stratospheric aircraft of extremely lightweight construction, configured for unmanned flight at very high altitudes. The stratospheric aircraft 2 according to Fig. 2 differs from that according to Fig.1 in the arrangement of its horizontal stabilizer 6 at the upper end of its vertical stabilizer 7. This allows the camera 9, again arranged near the center of mass 16 on the fuselage 3, to have a clear view of the wide wing leading edge 19 of the horizontal stabilizer 6, and this wing leading edge 19 is designed here as the mirrored outer surface 13, which redirects the field of view of the camera 9 toward the area 15 to be monitored. Given the large horizontal width of the wing leading edge 19, which makes a comparatively large aperture angle 11 of the original field of view 10 usable, the field of view 14, which is further expanded horizontally due to the increasing distance from the camera 9, is sufficient to capture the entire area 15 of the aircraft 1 to be monitored.The size of the leading edge 19 of the horizontal stabilizer 6, i.e., its horizontal extent, also allows its relative position to the camera 9 to be determined relatively easily from the images of the horizontal stabilizer 6 in the images of the camera 9. Horizontally, the wing leading edge 19 is not convexly curved. Accordingly, the aperture angle 17 of the expanded field of view 14 in the horizontal direction is not larger than the aperture angle 11 of the original field of view 10 of the camera 9. Vertically, the wing leading edge 13 of the horizontal stabilizer 6 is convexly curved, but the associated vertical expansion of the camera's field of view is not utilized here.

[0039] In principle, the reflective outer surface 13 can be an additional outer surface compared to a basic structure of the aircraft 1. Preferably, however, the reflective outer surface 13 is an existing surface of this basic structure, which is treated such that it has sufficient quality as an optical mirror. LIST OF REFERENCE SYMBOLS 1 aircraft 2 stratospheric aircraft 3 Hull 4 wings 5 half wings 6 horizontal stabilizer 7 Vertical stabilizer 8 tail boom 9 Camera 10 original field of vision 11 Opening angle of the original field of view 10 12 leading edge 13 reflective outer surface 14 widened field of view 15 area to be monitored 16 Center of mass 17 Opening angle of the expanded field of view 14 18 Optical Markers 19 Wing leading edge

Claims

[1] Method for optically monitoring an area (15) of an aircraft (1), - wherein images of the area to be monitored (15) are taken during the flight of the aircraft (1) with a camera (9) fixedly mounted on the fuselage (3) of the aircraft (1), and - and the images are analyzed, characterized by that a field of view of the camera (9) with a reflective outer surface (13) of the aircraft (1) is at least partially redirected towards the area (15) to be monitored. [2] Method according to claim 1, characterized by that a distortion of the images of the area to be monitored (15) due to the reflection on the reflective outer surface (13) is eliminated when the images are analyzed. [3] Method according to claim 2, characterized bythat positions of optical markers on the reflective outer surface (13) are determined in the images recorded with the camera (9) and are taken into account when calculating the distortion. [4] Method according to one of the preceding claims, characterized by that when analyzing the images, positions of optical markers (18) in the area to be monitored (15) are identified in the images of the camera (9). [5] Method according to one of the preceding claims, characterized by that movements of the camera (9) are detected and taken into account when evaluating the images from the camera (9). [6] An aircraft (1) having a fuselage (3), an area (15) to be monitored and a camera (9) fixedly arranged on the fuselage (3) of the aircraft (1), the field of view of which is directed at least partially onto the area (15) to be monitored, characterized bythat a reflective outer surface (13) of the aircraft (1) at least partially redirects the field of view of the camera (9) towards the area (15) to be monitored. [7] Method or aircraft (1) according to one of the preceding claims, characterized by that the reflective outer surface (13) is convexly curved. [8] Method or aircraft (1) according to one of the preceding claims, characterized by that the reflective outer surface (13) is a surface of a vertical stabilizer (7) and / or horizontal stabilizer (6) of the aircraft (1). [9] Method or aircraft (1) according to one of the preceding claims, characterized by that the camera (9) is arranged in the area of ​​the center of mass (16) of the aircraft (1). [10] Method or aircraft (1) according to one of the preceding claims, characterized bythat the camera (9) is arranged at a distance of at least 1 m, preferably at least 2 m and more preferably at least 3 m from the reflective outer surface (13). [11] Method or aircraft (1) according to one of the preceding claims, characterized by that the area to be monitored (15) is a half wing (5) or a wing (4) of the aircraft (1). [12] Method or aircraft (1) according to one of the preceding claims, characterized by that optical markers are applied to the reflective outer surface (13). [13] Method or aircraft (1) according to one of the preceding claims, characterized by that optical markers (18) are arranged in the area to be monitored (15). [14] Method or aircraft (1) according to claim 12 or 13, characterized by that the optical markers (18) have light sources. [15] Method or aircraft (1) according to one of the preceding claims, characterized by that an illumination light source aligned coaxially to the camera (9) is arranged on the camera (9). [16] Method or aircraft (1) according to one of the preceding claims, characterized by that an acceleration sensor is arranged on the camera (9). [17] Method or aircraft (1) according to one of the preceding claims, characterized by that the reflective outer surface (13) is mirror-polished and / or painted. [18] Method or aircraft (1) according to one of the preceding claims, characterized by that the aircraft (1) is an ultralight aircraft and / or a stratospheric aircraft (2).

Images