Method, aircraft, and system for detecting features of an object using first and second resolutions

Abstract

Embodiments according to the first and second aspects of the invention are based on the core concept of detecting features (110a, 450, 460) of an object (110, 400) and include the steps of flying over the object and detecting at least a part (110b) of the object with a capture unit (130) using a first resolution and providing a record of the area of ​​the object having the features using a second resolution, which is higher than the first resolution.

Description

Technical Field

[0001] Embodiments according to the present invention relate to a method, an aircraft, and a system for detecting features of an object. Further embodiments relate to the inspection of wind turbines by drones assisted by AI (Artificial Intelligence).

Background Art

[0002] Wind power plants form an essential part of sustainable energy supply. The use of the inexhaustible resource wind enables safe power production without emissions. Although energy generation of the type by converting mechanical energy into electrical energy does not pose a direct risk, it is still very important to regularly maintain and inspect the power plants themselves. Due to the high output and respective dimensions of modern wind power plants, modern wind power plants have to withstand very large mechanical forces over many years. Therefore, not only regarding the safety of the power plants, but also regarding the efficiency of the power plants, damage, for example, to the rotor of a wind turbine that can reduce the efficiency of the power plant, should be detected early enough to be eliminated.

[0003] One of the main problems is the size of modern wind power plants and the difficulty of accessing most of the surface of the power plants. Conventional methods include the use of climbers to inspect the power plants and record damage. Apart from the inherent danger to the climbers, such operations are costly due to the need for specially trained personnel and the loss of power production of each wind turbine during the entire inspection time. Moreover, due to the availability and speed of such climbers, sufficiently short inspection intervals for maintaining multiple power plants in a modern wind farm are impossible.

[0004] Considering this, there is a need for a concept that provides an improved trade-off between the speed and accuracy of inspections of wind power plants at low cost. Further, each concept shall constitute excellent scalability to ensure a sufficiently short inspection interval even for multiple power plants in a wind farm, for example, for legally required repeated inspections.

Summary of the Invention

Problems to be Solved by the Invention

[0005] The underlying object of the present invention is to provide a concept that enables the detection of object features at low cost, with little time and effort and high accuracy.

Means for Solving the Problems

[0006] This object is solved by the subject matter of the independent claims. Further developments of the invention are defined in the dependent claims.

[0007] An embodiment according to a first aspect of the present invention provides a method for detecting features of an object, the method comprising step (a) comprising flying along the object and optically detecting at least a part of the object at a first resolution by at least one capture unit to generate a plurality of images, each image representing at least partially different regions of the object. Further, the method comprises step (b) comprising evaluating the plurality of images to classify the generated images into images without features and images with features. Moreover, the method comprises step (c) comprising optically detecting again, at a second resolution higher than the first resolution, the region of the object where the assigned image has features.

[0008] A further embodiment according to a second aspect of the present invention provides a method for detecting features of an object, the method comprising step (a) of flying along the object and optically detecting at least a part of the object by means of at least one capture unit to generate a plurality of images, each image representing at least partially different regions of the object, and for one region, an image having a first resolution and a plurality of partial images each having a second resolution higher than the first resolution are generated. Moreover, the method comprises step (b) of evaluating the plurality of images in order to classify the generated images into images without features and images with features. Further, the method comprises step (c) of providing partial images of regions of the object for which the assigned images have features.

[0009] A further embodiment according to a first aspect of the present invention provides an unmanned aerial vehicle, such as a drone, for detecting features of an object using at least one capture unit for generating an image by optical detection. Here, the unmanned aerial vehicle can be controlled to fly along the object and optically detect at least a part of the object at a first resolution by means of the capture unit to generate a plurality of images, each image representing at least partially different regions of the object. Moreover, the unmanned aerial vehicle can be controlled to optically detect again at a second resolution higher than the first resolution a region of the object for which the assigned image has features.

[0010] A further embodiment according to a second aspect of the present invention provides an unmanned aerial vehicle, such as a drone, for detecting features of an object using at least one capture unit for generating an image by optical detection. Here, the drone can be controlled to fly along the object and optically detect at least a part of the object by means of the capture unit to generate a plurality of images, each image representing at least partially different regions of the object. Moreover, the unmanned aerial vehicle can be controlled to generate, for each region, an image having a first resolution and a plurality of partial images each having a second resolution higher than the first resolution.

[0011] A further embodiment according to a first aspect of the present invention provides a system for detecting features of an object using an unmanned aerial vehicle, such as a drone, the unmanned aerial vehicle being controllable to fly along the object to optically detect at least a part of the object at a first resolution by means of at least one capture unit to generate a plurality of images, each image representing at least a partially different region of the object. Further, the system is configured to evaluate the plurality of images to classify the generated images into featureless images and feature-containing images. Here, the unmanned aerial vehicle can be controlled to optically detect again a region of the object with features in the assigned image at a second resolution higher than the first resolution.

[0012] A further embodiment according to a second aspect of the present invention provides a system for detecting features of an object using an unmanned aerial vehicle, such as a drone, the unmanned aerial vehicle being controllable to fly along the object and optically detect at least a part of the object by means of a capture unit to generate a plurality of images, each image representing at least a partially different region of the object. Moreover, the unmanned aerial vehicle can be controlled to generate, for each region, an image having a first resolution and a plurality of partial images each having a second resolution higher than the first resolution. Further, the system is configured to evaluate the plurality of images to classify the generated images into featureless images and feature-containing images, and to provide, for example, partial images of a region of the object with features in the assigned image to classify or catalog the detected features.

[0013] Embodiments according to the first and second aspects of the present invention are based on the core idea of flying along an object to detect at least a part of the object at a first resolution using a capture unit, and for a region of the object with features, providing an image having a second resolution higher than the first resolution, for example, by detecting such a region again at the second resolution (first aspect), or by generating several partial images having a second resolution for each region and providing the partial images with features.

[0014] The object can be a large and difficult-to-access object such as a wind power plant, an oil platform, a bridge, a crane, a factory plant, a refinery, and a ship. The characteristics of the object can be, for example, damage to the object. According to an embodiment, the method of the present invention can be used to detect these damages in the form of, for example, deviations from geometric standards such as cracks, holes, curvatures, the occurrence of rust, or other indicators that can impair the structural integrity of the object.

[0015] For example, by flying along the object using an unmanned aircraft such as a drone, such an object can be detected with little time and resource effort. In particular, the employment of people for the detection itself, such as climbers in a wind power plant, can be omitted, thereby not only saving costs but also preventing people from being put at risk. Moreover, by flying along the object, parts of the object that are inaccessible by other methods can be detected. Detecting at least a part of the object at a first resolution and then re-detecting the area of the object at a second resolution higher than the first resolution enables, for example, the entire object not to be scanned at a higher resolution, which may result in a lot of time and data effort, and based on the detection at the first resolution, only areas where damage cannot be excluded or where there is a possibility of damage need to be scanned, thus enabling a very good trade-off between the speed and accuracy of feature detection. In other words, the approach of the present invention consists of generating a new image with a higher resolution of the area of the object containing defects or other features of interest after detecting and classifying an image of the object.

[0016] Briefly speaking, based on a plurality of images having a first resolution, a rough detection of an object or a feature of an object can be performed. These images can be evaluated and classified with respect to features such as damage in order to re-detect, at a second, higher resolution, the region of the object in which the assigned image contains the feature. Thus, based on an image having a second resolution, for example, small damages can be detected with high accuracy. The evaluation of the plurality of images can be performed, for example, by an automated method based on a technique from machine learning, or manually, for example, by a person. Further, for example, the plurality of evaluations can be performed in a partially automated method using, for example, an automated method that assists a human in the evaluation.

[0017] Here, in order to generate a new image using a second evaluation, a step of flying again along after the evaluation of the plurality of images on the ground can be performed, or the evaluation can be performed during flight, i.e., while flying along the object and optically detecting at least a part of the object. In order to generate a new image having a higher resolution, the distance of the capture unit to the object can be shortened, or the zoom setting of the capture unit can be changed. Moreover, for example, based on a previous image classification, a raster image can be generated, and each partial image of the raster image has a second, higher resolution.

[0018] Furthermore, for example, regardless of the evaluation of an image regarding features, for an image of the detection of a part of an object having a first resolution, a plurality of partial images each having a second resolution can be generated. Thus, when flying once along the object, for example, the entire amount of data for the evaluation regarding damage to the object can be generated. This has an advantage, for example, when a drone that flies along the object and has a small computing power configured to detect and store a large amount of data but is not sufficient to evaluate the image. Further, an integral scan of the object can be performed, and this integral scan consists of, for example, a plurality of images having a first resolution, and a plurality of partial images having a second resolution are provided for a plurality of images having a first resolution or even all of the images so that a dataset having multi-stage accuracy for the description of the object is available. This may have an advantage, for example, in certain safety-critical applications where an uninterrupted proof regarding the state of the object is required. Further, such a dataset can, for example, provide an intuitive option for the evaluation of a person. To inspect for damage, a person can, for example, quickly and more detailedly examine the parts of the object by the partial images having a second resolution stored for a plurality or even all of the images having a first resolution, and the important regions of the object, from the appearance of the object in the form of a 3D model consisting of images having a first resolution.

[0019] Similar to the above description, a plurality of images having a first resolution and a plurality of sub-images having a second resolution can be evaluated, for example, in an automated method, in order to classify the generated images into images without features and images with features, such that, for example, sub-images of regions of an object whose assigned image contains features can be provided and, for example, highlighted for a person. Briefly, an image having a first resolution can be generated together with a high-resolution raster image having a second resolution, the images can be classified, and based on that classification, a raster image of a sub-image, for example an image representing a defective region, can be provided. It should be noted here that the evaluation of the plurality of images and / or sub-images can be done in an automated method, or a partially automated method, or a manual method.

[0020] In other words, during a first time of flight along, in addition to the first image required for evaluation, for example evaluation on the ground by a laptop, for each image, for example automatically, a raster image having a higher resolution, for example a plurality of sub-images each having a second resolution, can also be generated. When using an unmanned aerial vehicle, for example a drone, comprising a capture unit, the drone can stop at a waypoint of its flight trajectory, generate the first image, and then immediately generate a high-resolution raster image from the same position. To increase the resolution, for example, a zoom lens can be used. Furthermore, to increase the resolution, the capture unit can be equipped with a plurality of cameras, such as two or three cameras. Moreover, the capture unit can be equipped with a plurality of lenses or camera lenses (multi-lens camera) for increasing the resolution. After classifying the images with features, the high-resolution images with features can be directly represented from the raster images. This means that no second flight needs to be performed since all the necessary images with the highest resolution already exist and only a selection needs to be made.

[0021] In addition, for example, an automatic evaluation of an image, for example, an image having a first resolution or an image having a second resolution, can be performed during or after flight along an object. Depending on the application or the existing hardware, for example, immediately after detecting an image having a first resolution during flight, it is determined whether a further image of the relevant region of the object is detected at a second, higher resolution by evaluating or classifying the image. Further, as discussed above, and for at least a part of the object, for both one or more images having a first resolution and the relevant one or more images having a first resolution, a plurality of images having a second, higher resolution can be generated, and the evaluation or classification can be performed during flight, that is, during or after flight along, based on any combination of images.

[0022] The evaluation or classification can also be performed after flying along the object, that is, after flying along the object to generate an image having a first resolution and before flying along the object again to generate an image having a second resolution, on an external computer, that is, a computer that is not part of an unmanned aerial vehicle such as, for example, a capture unit or a drone, for example, on a laptop.

[0023] Thus, the method according to the invention enables an accurate scan of an object with little time and resource effort.

[0024] In a further embodiment according to a first aspect of the invention, step (b) is performed after flying along the object and step (c) includes the step of approaching the region of the object whose assigned image contains features. For example, by an automatic evaluation of the image after flying along the object, preprocessing can be performed based on the image having the first resolution from step (a), and based on this preprocessing, step (c) can be performed such that, for example, only the region of the object whose assigned image contains features is detected again at the second resolution. Thereby, significant time and data savings can be achieved.

[0025] Moreover, for example, by automatically evaluating a plurality of images after flying along an object, the evaluation can be performed on an external computer, for example, so that a drone for flying along the object only needs to meet low hardware requirements. Further, especially for large objects, it may be necessary to recover the flight ability of the aircraft after flying along, and this can be performed simultaneously with the evaluation in a time-saving manner. Step (c) can be further performed after step (b), for example, after the intermediate landing discussed above. Briefly, the evaluation of the images can be performed after flying, and then fly along the object again to generate an image of the defective area at a higher resolution.

[0026] In an embodiment according to the first aspect of the present invention, the capture unit generates one image having the same focal length in steps (a) and (c) respectively. Further, the object is approached in step (a) such that the object has a first distance with respect to the capture unit when the capture unit generates an image, and the object is approached in step (c) such that the object has a second distance shorter than the first distance with respect to the capture unit when the capture unit generates an image.

[0027] Thereby, a simple and cost-effective capture unit having only a single focal length can be used. Therefore, increasing the resolution is obtained by shortening the distance from the capture unit to the object in step (c).

[0028] In an embodiment according to a first aspect of the present invention, the object is approached in steps (a) and (c) such that the capture unit has the same or a similar distance to the object when generating an image. Further, the capture unit generates an image having a first focal length in step (a) and an image having a second focal length in step (c), the second focal length being longer than the first focal length. Thus, increasing the resolution can be done by changing the focal length of the lens of the capture unit, for example, by changing the zoom setting or by replacing the lens. The capture unit can be converted during an intermediate stop, for example, to detect an image having a second focal length in step (c). Further, the capture unit can also comprise cameras with different focal lengths such that steps (a) and (c) can also be performed during a single flight.

[0029] In an embodiment according to the first aspect of the invention, in step (a), the capture unit uses at least one of a first camera having a first focal length, a first lens having a first focal length, and / or a first camera lens having a first focal length, or a zoom lens having a first zoom setting corresponding to the first focal length. Moreover, step (c) includes the step of replacing the first camera of the capture unit with a second camera having a second focal length, and / or the step of replacing the first lens of the capture unit with a second lens having a second focal length, and / or the step of replacing the first camera lens of the capture unit with a second camera lens having a second focal length, or the step of setting the zoom lens of the capture unit to a second zoom setting corresponding to the second focal length. Both the step of changing the camera, lens, and / or camera lens and the step of changing the zoom setting can be performed during mid-air landing or during flight along an object, for example, in an automated manner. By changing at least one of the camera, lens, camera lens, and / or zoom setting, for example, when flying along an object in an automated or autonomous manner, increasing the resolution can be performed without changing the detection distance, i.e., the distance from the capture unit to the object, so that there is no need to change the waypoints that determine the flight trajectory along it.

[0030] In an embodiment according to the first aspect of the invention, the step of optically redetecting the area in step (c) includes the step of generating a plurality of partial images of the area, each having a second resolution. Here, step (c) can include, for example, in particular, the approach described above of the relevant image over the area of the object containing the features. Briefly, a raster image with a higher resolution of each area can be generated. In this way, damage can be detected reliably and accurately.

[0031] In an embodiment according to the first aspect of the present invention, the position and / or location information of the capture unit is assigned to each image generated in step (a). Further, in step (c), the area of the object to be flown along is determined by using the position and / or location information of the image containing the features. Briefly, waypoints for flying again along or approaching can be generated based on the position / location information of the capture unit. In that way, when approaching the object again, for example, a trajectory can be generated that includes only waypoints suitable for detecting areas including potential damage. Therefore, the step of approaching the object again or flying again along the object can be performed with little time and effort to generate an image having a second resolution.

[0032] In an embodiment according to the first aspect of the present invention, the step of flying along the object is performed using an unmanned, crewless, or pilotless aircraft, a UAV, for example, a drone including a capture unit. Further, step (b) includes the step of transmitting the image generated in step (a) from the unmanned aircraft to a computer such as a laptop computer, and the step of evaluating the image by the computer. Here, the step of evaluating the image includes the step of evaluating the image in an automated method. For example, part of the evaluation is performed in an automated method, or the entire evaluation of the image is performed in an automated method. With sufficiently high-speed data transmission between the computer and the aircraft, the step of re-optially detecting in step (c) can be performed during the flight along in step (a) so that the step of selecting the area of the object detected at the second resolution can be performed based on the image having the first resolution during the flight along. By transferring the evaluation to an external computer, for example, a remote computer, an aircraft with low hardware requirements can be used. Moreover, for example, the computer can also transmit and evaluate the image during landing to prepare for approaching the object again to generate an image having a second resolution.

[0033] In an embodiment according to the first aspect of the present invention, the unmanned aircraft autonomously flies along an object in step (a). Further, step (b) includes generating waypoints using the position and / or location information of the image including the features, and transmitting the waypoints from the computer described above to the unmanned aircraft, for example. Moreover, in step (c), the unmanned aircraft autonomously approaches the area of the object using the waypoints. Thereby, a completely autonomous detection of the object or the features of the object can be performed. The waypoints can plan a time-efficient trajectory by which each area of the object can be detected at a second resolution in step (c). The waypoints can be created, for example, based on the CAD model of the object. Thereby, the model can be improved by detecting the object, and based thereon, the waypoints can be adapted.

[0034] In an embodiment according to the first aspect of the present invention, the step of flying along an object by an unmanned aircraft including a capture unit, for example a drone, is performed autonomously. Moreover, the unmanned aircraft includes a computer, and step (b) includes generating waypoints by the computer of the unmanned aircraft using the position and / or location information of the image including the features by evaluating the image. Here, the step of evaluating the image includes an automatic evaluation of the image, and the evaluation can be performed, for example, in a completely automated or partially automated manner. In addition, in step (c), the unmanned aircraft autonomously approaches the area of the object using the waypoints. By providing the computing power for evaluating the image and generating the waypoints by the unmanned aircraft, the detection of the object or the features of the object can be performed by flying along once, thereby in a time-efficient manner. The step of generating the waypoints can also include adapting the existing waypoints such that the unmanned aircraft adapts its own flight trajectory based on the evaluation of the image while capturing the image.

[0035] In an embodiment according to the first aspect of the present invention, steps (a) to (c) are performed while flying along an object such that in step (a), an image of a region is generated, and in step (b), the image generated in step (a) is classified for a further region before generating an image. Moreover, steps (a) to (c) are such that in step (b), when the image is classified as including features, in step (c), before an image is generated for a further region and before generating a further image, the area is optically detected again, and in step (b), when the image is classified as not including features, it is performed during flight along the object so that an image is generated for a further region. Thus, once flying along an object, features or the object can be detected. By classifying the image generated in step (a), it is possible to prevent capturing an image having a second, higher resolution that does not include, for example, damage to the object, so that only relevant information is detected. In other words, the evaluation of the image can be performed during flight, and for example, an image having a higher resolution of only the defective region can be generated.

[0036] In an embodiment according to the first aspect of the present invention, the capture unit generates images having the same focal length in both steps (a) and (c). Moreover, the object is approached in step (a) such that the capture unit has a first distance to the object when generating an image. Further, in step (c), the distance of the capture unit to the object is reduced to a second distance that is shorter than the first distance. Briefly speaking, a higher resolution is obtained by a shorter distance to the object. Reducing the distance can be easily and quickly achieved by adapting the flight trajectory.

[0037] In an embodiment according to the first aspect of the present invention, the object has a first distance with respect to the capture unit when the capture unit generates an image, and is approached in step (a) such that the capture unit generates an image having a first focal length. Further, in step (c), the distance of the capture unit with respect to the object is the same as or similar to the first distance, and the capture unit generates an image having a second focal length longer than the first focal length. Thus, for example, when autonomously detecting an object, a predetermined waypoint can be maintained so that no additional adaptation of the flight trajectory is required. Here, for example, the period during which the unmanned aircraft is at each waypoint can simply be made longer to generate an image having a second resolution. The distance of the capture unit with respect to the object in step (c) can be similar or even the same compared to the first distance such that the improvement in resolution due to the change in focal length is dominant over the improvement in resolution due to the change in distance. These two distances can deviate from each other by a few percent, for example, less than 5%, or less than 10%, or less than 20%. However, it should be noted that a predetermined waypoint or flight trajectory can also be changed or adapted according to the embodiment.

[0038] In an embodiment according to the first aspect of the present invention, the capture unit includes at least one of a plurality of lenses, a zoom lens, a plurality of cameras, and a plurality of camera lenses. Further, in step (a), the capture unit uses a first camera, a first lens, and / or a first camera lens having a first focal length, or sets the zoom lens to a first zoom setting according to the first focal length. Moreover, in step (c), the capture unit uses a second camera, a second lens, and / or a second camera lens having a second focal length, or sets the zoom lens to a second zoom setting according to the second focal length. These adaptations can be performed in an automated manner during flight, for example, such that features of an object can be detected by flying along the object once. The zoom setting, and / or the selection of the camera, lens, and / or camera lens can be linked to the waypoint and / or the timing of the flight.

[0039] In an embodiment according to the first aspect of the present invention, the step of optically detecting the region again in step (c) includes the step of generating a plurality of partial images of the region, each having a second resolution. Thus, put simply, the plurality of partial images can form a raster image having a higher resolution, for example, so as to be assigned to an image having a first resolution and generate a plurality of detailed images.

[0040] In an embodiment according to the first aspect of the present invention, the step of flying along an object is autonomously performed by an unmanned aerial vehicle, such as a drone, including a capture unit. Further, the unmanned aerial vehicle includes a computer, and step (b) includes the step of evaluating an image by the computer of the unmanned aerial vehicle. Here, the step of evaluating the image includes the step of evaluating the image in an automated method, and thus, the evaluation can be performed, for example, in a fully automated method or in a partially automated method. By providing computing power by the aircraft, for example, features of an object can be detected by flying along the object once so that the method can be performed in a partially time-efficient method. Moreover, due to the computing power, the aircraft can adaptively plan its trajectory based on the evaluation result, for example, in the form of waypoints. Thus, a fully autonomous detection of the features of an object can be performed.

[0041] An embodiment according to the first aspect of the present invention includes the following step (d), and step (d) includes the step of transmitting the image generated in step (c) to an evaluation unit, for example, to classify or catalog the detected features. The higher-resolution image generated in step (c) can be transmitted to an evaluation unit, such as an external computer, to evaluate features of an object, such as damage to the object, such as cracks. This information can then be input, for example, into an existing model of the object. Further, the evaluation unit can also, for example, detect a specific type of damage, approach the damage again, and send back further instructions to the unmanned aerial vehicle if, for example, other measurement methods are used to detect the damage. In the case of a wind power plant, for example, electrical measurements of the lightning protection device of the wind power plant can be performed based on image evaluation.

[0042] In an embodiment according to a second aspect of the present invention, the object is approached in step (a) such that the capture unit has a certain distance to the object when generating the image and the partial image. Further, in step (a), the capture unit generates an image having a first focal length and a partial image having a second focal length that is longer than the first focal length. In other words, increasing the resolution is done by changing the focal length.

[0043] In an embodiment according to a second aspect of the present invention, the capture unit includes a zoom lens that uses a first zoom setting corresponding to the first focal length when generating an image and a second zoom setting corresponding to the second focal length when generating a partial image. Changing the zoom setting can be performed during flight such that both images having a first resolution and a second resolution can be taken from the same waypoint of the flight trajectory. Thus, this method of generating a partial image can be performed in a particularly time - efficient manner.

[0044] In an embodiment according to a second aspect of the present invention, the step of flying along an object with an unmanned aerial vehicle, for example a drone, including the capture unit, is performed autonomously. Moreover, step (b) includes the step of transmitting the image and the partial image generated in step (a) from the unmanned aerial vehicle to a computer such as a laptop computer, and the step of evaluating the image by the computer. The step of evaluating the image includes an automatic evaluation of the image, and the evaluation of the image can be performed, for example, in a fully automated or partially automated manner. Further, step (c) includes the step of providing a partial image of the image assigned to the region by the computer. The evaluation can be performed, for example, by a machine - learning method. Briefly speaking, the method of the present invention generates a classified image and a detailed image in the form of a partial image and provides them by a computer. Thereby, an intuitive evaluation or assessment of features, for example damage, can be performed.

[0045] In an embodiment according to a second aspect of the present invention, the step of flying along an object is autonomously performed by an unmanned aerial vehicle including a capture unit, for example a drone. Moreover, the unmanned aerial vehicle includes a computer, and step (b) includes the step of evaluating an image and a partial image by the computer of the unmanned aerial vehicle. The step of evaluating the images (B1 to B4) and the partial images (B11 to B44) includes an automatic evaluation of the images (B1 to B4) and the partial images (B11 to B44). The images and partial images can be evaluated, for example, in a partially automated or fully automated manner. Further, in step (c), the unmanned aerial vehicle transmits the partial image to an evaluation unit, for example, to classify or catalog the detected features. By evaluating the images and partial images by the unmanned aerial vehicle while flying along the object, the features of the object can already be detected with reduced time and labor. The classified or cataloged features are then provided by the evaluation unit, for example, directed to a person evaluating, and the person evaluating can, for example, during flight of the aircraft, restart further detection of the relevant area by, for example, non-optical measurement methods (for example, lightning protection measurement and / or humidity measurement).

[0046] In an embodiment according to the first and / or second aspect of the present invention, the features to be detected include defects of the object or predetermined elements of the object. The defects can be, for example, cracks, holes, occurrence of rust, damage to the paint, or other optically detectable surface changes. Further, the features can also be predetermined elements of the object, such as characteristic geometric shapes of the object, for example in the case of a wind turbine, the blade tip or the rotor flange, or specific devices such as rivets or screws. In some applications, it may be advantageous to detect each predetermined element as accurately as possible, for example, to construct a 3D model of the object as accurate as possible or to eliminate damage and deviation from the specification.

[0047] In an embodiment according to the first and / or second aspect of the present invention, step (b) includes AI or machine learning. Such a method can, for example, rapidly evaluate or classify an image with little resource effort regarding the presence of features of an object. Moreover, the method of the present invention using known reference objects or known reference features can be used to generate training data for each method. AI or machine learning can be used by itself or to assist a human. Such a method enables a huge time saving, especially with respect to a large number of images to be evaluated for large objects such as oil platforms or wind turbines.

[0048] In an embodiment according to the first and / or second aspect of the present invention, the object includes an energy generation plant, for example, a wind power plant or a solar power plant, or an industrial plant, for example, an oil platform, a factory plant, a refinery, or a building, for example, a multi-story building, or an infrastructure means such as a bridge. Furthermore, the object can also be a crane. In particular, for example, with respect to an industrial plant, the method of the present invention can be executed during operation without exposing a human to danger or without the need to stop the operation.

[0049] In an embodiment according to the first aspect and / or the second aspect of the present invention, an unmanned aircraft, for example a drone, is configured to transmit a plurality of images to an external computer, for example a laptop computer, which classifies the generated images into images without features and images with features. Further, the unmanned aircraft is configured to receive from the external computer information indicating an area of an object to be optically detected at a second resolution. Based on the information of the external computer, the unmanned aircraft can accordingly generate an image of each area at the second resolution. The communication as well as the use of the information can be carried out during flight, for example while flying along an object to generate an image having a first resolution of the object, or during an intermediate stop between, for example, the step of flying along an object to generate an image having a first resolution and the step of approaching the object again to generate an image of a selected area of the object at a second resolution. Further, the intermediate stop can be used to restore the flight ability of the unmanned aircraft, for example to replace the battery of the drone. Moreover, the computer can also provide a trajectory plan for the unmanned aircraft, for example for autonomous flight, and can form the evaluation unit discussed above.

[0050] In an embodiment according to the first and / or second aspect of the present invention, an unmanned aircraft, for example a drone, comprises a computer configured to evaluate a plurality of images in order to classify the generated images into images without features and images with features. Thereby, the drone can be independent of a sufficiently fast communication connection with an external computer. Further, by evaluating a plurality of images during flight, time savings can be achieved.

[0051] Embodiments according to the present invention will be discussed in more detail below with reference to the accompanying drawings. With respect to the schematic diagrams shown, it should be noted that the functional blocks shown are to be considered not only as elements or functions of the device of the present invention, but also as respective method steps of the method of the present invention, and that each method step of the method of the present invention can also be derived therefrom.

Brief Description of the Drawings

[0052]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

DETAILED DESCRIPTION OF THE INVENTION

[0053] Before discussing the embodiments of the present invention in more detail below with reference to the drawings, it should be noted that the same, functionally equivalent, or equal elements, objects, and / or structures in different figures are given the same or similar reference numerals so that the descriptions of these elements shown in different embodiments are interchangeable or applicable to each other.

[0054] FIG. 1 shows an object 110, an unmanned aircraft 120 having a capture unit 130, and a computer 140. Here, the aircraft 120 should be considered merely as an option, for example, as a possible implementation form of a movable capture unit in the form of a drone having a camera.

[0055] Starting from the starting point S, the aircraft 120 flies along an object to detect one or several features 110a. The flight trajectory is indicated by waypoints WP1 to WP4. The flight trajectory can result from, for example, a previous trajectory plan. Here, for example, the generation of waypoints can be executed based on a 3D model of the object provided to the aircraft 120 via connection 140a by, for example, a computer 140. It should be noted here that the computer 140 can be part of the aircraft 120 such that the aircraft autonomously plans its own trajectory, for example. Furthermore, the trajectory can also be manually determined in advance by a human pilot, for example, due to the absence of a 3D model of the object 110.

[0056] When flying along the object 110, the capture unit 130 detects the front face 110b of the object 110, or generally a part of the object. Here, the capture unit 130 generates a plurality of images B1 to B4 having a first resolution, and each image represents at least a partially different region of the object 110, or the front face 110b of the object. Thus, the images B1 to B4 can overlap partially as shown in FIG. 1, or in other words, the images B1 to B4 can include information regarding partially equal image sections or partially equal regions of the object. Furthermore, the images may not overlap. As an example in FIG. 1, each of the waypoints WP1 to WP4 is associated with one of the images B1 to B4. Briefly speaking, a plurality of waypoints can be determined from where the object was detected, for example, where it was captured.

[0057] According to the present invention, in order to detect the feature 110a, a plurality of methods and method steps are available that are mutually interchangeable and can be used together in any combination, unless the feature specifically states otherwise. Some options of the present invention are discussed below with the help of FIGS. 2 and 3. However, this does not represent a limited list of method steps and is merely useful for improving the understanding of the concept of the present invention and its configuration.

[0058] Figure 2 shows images B1 to B4 having the same resolution. Images B1 to B4 are classified (140c) into an image 210 that does not include feature 110a, including images B1, B3, and B4, and an image 220 that includes feature 110a, including image B2. The classification is implemented using computer 140 as an example in FIG. 2, but can be executed by any classification unit. The classification can be implemented particularly by a machine learning method. The classification can be executed, for example, by artificial intelligence (AI).

[0059] Again, as shown in FIG. 1, the result of the classification can be communicated (140b) to the aircraft 120 or the capture unit 130 during flight, or can be communicated (140a) after flying along the object 110 and then landing at the starting point S. Again, it should be noted that the computer 140, or each classification unit, can be part of the aircraft 120 such that each communication 140a, 140b is considered optional. Further, the bi-directionality of the communications 140a, 140b shown in FIG. 1 is also optional, and any combination of information can be transmitted to the computer via the aircraft or vice versa. Depending on the existing hardware, a plurality of possible task assignments (e.g., regarding image classification, waypoint calculation, feature classification) can be realized between the aircraft 120 and the computer 140, and the communications 140a, 140b can be configured accordingly.

[0060] Based on the classification, the capture unit 130 can optically detect again the region of the object 110 including the feature by the image B2 assigned to the region where the feature 110a is detected at a second resolution higher than the first resolution.

[0061] When the classification is performed while flying along the object 110, the aircraft can generate the image B2, detect the features on the image B2, and then stay on the waypoint WP2 and can detect each region of the object again. However, the aircraft 120 does not have to stay on the waypoint WP2 and can, for example, simply maintain the same or a similar distance from the object 110. To increase the resolution when detecting again, the capture unit 130 can increase the focal length, for example, by adapting the zoom setting or by changing the lens. Thereby, for example, an image B21 with a higher resolution, which is a partial image of the image B2, can be generated. Therefore, apart from the evaluation regarding the classification of the images B1 - B4 including the feature 110a, the position of the feature can be evaluated by using the position and location data of the aircraft 120 in order to direct the capture unit 130 towards the feature 110a to generate the image B21. Here, it should be noted that the image B21 does not necessarily have to include a partial region of the image B2. The region of the object detected in the image B21 can be selected, for example, depending only on the position of the feature 110a such that the image section of the image B21 is selected independently of the regions of the images B1 - B4.

[0062] For example, if the position of the feature 110a is unknown, a certain amount of partial images B21 - B24 can be generated at the second resolution. Furthermore, independently of the classification and detection of the feature 110a on the images B1 - B4, a plurality of regions of the object, or for example each region of the object, can be detected by a plurality of partial images of the regions (for example, for the image B1, partial images B11 - B14, for the image B2, partial images B21 - B24, etc.), each having the second resolution.

[0063] Each partial image can be transmitted, for example, via the communication 140b to an evaluation unit in the form of, for example, a computer 140 in order to classify or catalog the detected feature 110a.

[0064] Alternatively, the classification of images B1 - B4 can also be performed after flying along waypoints WP1 - WP4 and then landing at the starting point S. Subsequently, via communication 140a, a second flight trajectory 150 can be provided to the aircraft 120 based on the classification in the assigned position and / or location information regarding feature 110a. For easier understanding, FIG. 1 shows a trajectory 150 that flies from the starting point S to waypoint WP2 and returns to the starting point S. At waypoint WP2, the detection of the region of the object discussed above is performed again. Here, it should be noted again that the region of the object including feature 110a can be captured in a single image B21 or, for example, in a raster of sub - images of image B2 including sub - images B21 - B24 having a second resolution. Increasing the resolution can be obtained again by changing the zoom setting or by changing the lens of the capture unit 130. These adaptations can be performed manually, for example, during landing, after flying along the object 110 for the first time, and before approaching the object 110 via the flight trajectory 150. The flight itself can be performed autonomously or manually again.

[0065] To increase the resolution when detecting the object again, the aircraft 120 or the capture unit 130 can also shorten the distance d to the object. The same will be discussed below with reference to FIG. 3. FIG. 3 shows a part of FIG. 1 having the object 110, the aircraft 120, the capture unit 130, and the computer 140. Waypoint WP2 has a first distance d to the object 110. 1 For re - detection, the aircraft 120 can approach the object on waypoint WP2a such that the distance is shortened to distance d. 2 Thereby, the object 110, or the region of the object comprising feature 110a, can be captured at a higher resolution, for example, without changing the zoom setting or without changing the lens.

[0066] Adapting the trajectory of the aircraft 120 can be done during flight, as optionally shown in FIG. 3. After classifying the image B2 by communication 140b with the computer 140, the trajectory can be changed during flight. Here, the capture unit or the aircraft can communicate the image B2 to the computer 140 and then receive a new waypoint WP2a. Alternatively, the above-mentioned aircraft can itself include the computer 140 and perform the classification and trajectory adaptation itself. Here, any combination is possible such that the aircraft or the capture unit can, for example, perform the classification individually, communicate the classification result to an external computer, and instead receive a waypoint. Each trajectory adaptation involving a shortening of the distance can also be done during an intermediate landing via the communication 140a of FIG. 1 according to the present invention. Further, the flight trajectory can also consist of waypoints having different distances to the object 100 a priori, such that, for example, independently of the classification of the image or the detection of the feature 110a, a raster of partial images (B11 to B14, B21 to B24, etc.) is generated at a second resolution for a plurality of regions of the object by shortening the distance.

[0067] A further embodiment involves AI-assisted inspection of a wind turbine by a drone, which is discussed below based on FIG. 4. FIG. 4 shows a wind power plant 400 having a tower 410, a pod 420, a rotor 430, a rotor or blade 440, a rotor blade or blade tip 450, and a rotor blade flange 460. One of the rotors 440 has a damage 110a. This damage 110a or defect can be, for example, a crack. Regarding defect detection, the damage 110a can be a feature of the wind power plant to be detected. For example, regarding the generation of a model of the wind power plant 400 by a calibration flight of the drone, the blade tip 450 and / or the rotor blade flange 460 can be features of the wind power plant 400 to be detected. From the detection, a 3D model of the wind power plant can be generated, for example, via the known position of the aircraft at the time of detection and the mapping shape. From this 3D model, waypoints for autonomous inspection flight can be generated again. Moreover, a detected image 210 without features, a detected image 220 including features in the form of an image 220a where the feature is the damage 110a, and an image 220b where the feature is alternatively or additionally the blade tip 450 are shown.

[0068] Regarding the case where the feature is the damage 110a, a partial image 470 is shown as an example for illustration. According to an embodiment, the region having the image 220a including the feature can be detected again at a higher resolution, but it is not necessary to scan the entire previously scanned region of the object again, and only a partial region of the original image section or a region of the object can be detected. Here, when compared with FIG. 1, it should be noted that generally, in an embodiment, it is not necessarily the case that the image Bx is completely divided into the partial image Bxx by a new detection with an increased resolution. Furthermore, each partial image 470 does not have to completely exist within the image having the first resolution at which the position of the feature was detected from its evaluation. Briefly speaking, for comparison, "raster detection" is shown for detecting the blade tip 450 for the image 220b.

[0069] The basic concept of defect detection or pattern detection according to the embodiment is performed by detecting and then excluding defect-free areas with the help of AI. In other words, defect detection, i.e., the detection of, for example, damage 110a, is performed by detecting areas without defects (image 210), which are called patterns in this context, and by this pattern detection, for example, defect 110a can be inferred. Then, the areas or patterns of the wind power plant 400 that are detected by AI as not defect-free, i.e., having defects, and thus not excluded, are approached again to generate a high-resolution defect image. To approach again, automatic generation of waypoints can be used. Referring to FIG. 2, the objects related to the image 210 or the areas of the wind power plant 400 are excluded, and the areas related to the image 210 are approached again. Referring to FIG. 1, the high-resolution defect image can be, for example, an image B21 having a second resolution, or a plurality of partial images B21 to B24. As already discussed above, the generation of the high-resolution defect image can be performed by using a zoom lens and / or by further approaching the areas detected as not defect-free. Further, it is also possible to replace the lens of the capture unit.

[0070] Regarding the wind power plant 400, AI-assisted image / defect detection or pattern detection can be used, for example, in the following tasks or missions. 1. Calibration flight 2. Tower inspection 3. Blade inspection

[0071] According to the embodiment, AI assistance can be used in several stages. As an example, three stages are described below, but the features of the individual stages can be mutually exchanged or combined in any way unless otherwise indicated. They are merely for explaining the idea regarding the use of AI for feature detection and should thus not be considered limiting.

[0072] Stage 1: After the drone lands and transmits the image data, further processing of the image data by AI is then executed, and this processing is calculated or executed on a remote computer (laptop). The AI generates, for example, inspection after a calibration flight or waypoints for an inspection flight, or waypoints for approaching the wind farm 400 to detect a defect 110a or another feature (e.g., rotor tip 450) due to a defective flight, i.e., at a resolution raised after an inspection flight.

[0073] The drone autonomously executes an inspection flight and, after landing, transmits the images to the remote computer, for example, during battery replacement. Image / defect detection or pattern detection is performed on the remote computer. As a result, waypoints, for example, for a subsequent inspection flight based on a generally generated CAD model of the wind farm after a calibration flight, or for a subsequent defective flight, i.e., for approaching the detected defect 110a at a short distance, for example, and capturing the defect 110a at a high resolution, are resent to the drone after calculation.

[0074] Stage 2 Local Intelligence: The AI is executed or calculated in real time on an additional computing unit on the drone and controls the inspection flight after calibration or the approach to a defect during an inspection flight. The additional computing unit can be, for example, an add-on GPU (graphics processing unit) board (additional graphics processor board), or a CPU (central processing unit), or a specific AI board. Further, the computer described above can also be part of the drone and, thus, can provide the hardware for operating the AI.

[0075] The drone is equipped with its own local intelligence, for example, by an additional computing unit, and performs the calculations on-board, locally on its own drone hardware, for example, in real time. Moreover, the drone can directly perform actions during the calibration flight, such as calculating waypoints for a subsequent inspection flight, and can, for example, immediately thereafter, further perform an inspection flight. Defects are detected in real time, and it is done to immediately approach or zoom in directly on the detected defects and capture the defects at their respective high resolutions. Thereafter, the inspection flight continues until the next defect.

[0076] By using the 300 DJI Drone 1 with a P1 full-format camera and a 50 mm lens, or alternatively a zoom lens, the inspection flight is performed, for example, at a distance of 8 m up to the blade 440 and can approach the detected defect again at a distance of 3 m.

[0077] After the inspection flight is completed, the drone transmits data, for example, all the image data of the inspection flight having, for example, a low resolution or a first resolution (for example, corresponding to images B1 - B4 in FIG. 1), and all the generated defects or partial images having, for example, a high resolution or a second resolution (for example, one or some of the partial images B11,..., B44 in FIG. 1) to the cloud or an external computer. From the defect images, the defect 110a can be classified by the operator, and a defect protocol can be generated interactively.

[0078] Stage 3 Evaluation: The detected high-resolution defect images are classified by AI. The defect images stored in the cloud can be automatically classified with the help of AI, and a defect protocol can be automatically generated. According to various stages of AI assistance, AI can be used in the above three tasks as discussed below.

[0079] 1. Calibration flight: In an embodiment, the basic principle of AI assistance is the optical recognition and detection of the blade tip 450 by AI, and the calculation of the position of the blade tip 450, as well as the optical recognition and detection of the blade flange 460 by AI, and the calculation of the position, distance, and angle of the blade flange 460.

[0080] Alternatively or additionally, the pitch angle of the blade can be detected and / or calculated. Using these values, a final calculation or correction of a general model of the wind power plant 400, such as a CAD model, including the position and orientation of the power plant and the bending of the blade 440, can be performed. From these data, waypoints for the inspection flight can be calculated. This can be done using intermediate landings (remote - stage 1), or in real - time without intermediate landings (local intelligence - stage 2).

[0081] 2. Tower inspection: In tower inspection, the use of AI can be particularly advantageous for a large number of images. Based on the above - mentioned hardware (a 300 DJI drone with a P1 full - format camera and a 50 mm lens, or alternatively a zoom lens), for example, 400 images can be generated at a tower height of 145 m, at a resolution of 1.25 pixels / mm, at a distance of 9 m between the capture unit and the wind power plant. On the other hand, the possible variations of damage 110a, or in other words, defect classes, are manageable, and the damage, being mostly large - scale, can be reached relatively quickly by AI.

[0082] 3. Blade Inspection: The method of the present invention for blade inspection is, for example, similar to the method for tower inspection in a significantly smaller number of images. For example, to obtain a required or advantageous resolution of approximately 1.6 pixels / mm for the first inspection, for example, at a drone or capture unit distance of 7 m with respect to the blade, for example, at a blade distance of approximately 7 m, for example, for the use of AI, approximately 25 images may be generated or required per side. To obtain an improved resolution, for example, the second resolution discussed above, for example, a resolution higher than 3.5 pixels / mm required by the reviewer, the detected defect should be approached again or immediately at a distance of approximately 3 m with respect to the blade. However, as explained above, a change in the zoom setting or a change in the lens used may be performed to obtain an improvement in resolution.

[0083] Based on the following table, aspects of embodiments according to the present invention are briefly summarized again, and their advantages are shown based on numerical examples. The numerical values are based on a 300 DJI drone having the P1 full-format camera and a 50 mm lens described above. The image sensor has a width of 35.9 mm with 8197 pixels and a height of 24 mm with 5460 pixels.

[0084]

Table 1

[0085] In the table, the possible tasks or missions of the method of the present invention for the wind power plant described above are plotted in the form of tower inspection, flight inspection, and calibration (calibration flight). Further, examples of defect flights described above are entered. For each of these tasks, in the fourth column, the distance of the aircraft or drone with respect to the wind power plant, in the fifth column, the width of the image covered by each image section with respect to the surface of the wind power plant in mm, in the sixth example, the height of the image covered by each image portion with respect to the surface of the wind power plant in mm, and in the seventh column, the respective resolution in pixels / mm are entered.

[0086] The inspection of a wind power plant can be started, for example, with a calibration or a calibration flight. For this purpose, the pilot flies along the wind power plant at a distance of 25 m with an aircraft, for example a drone. Here, the wind power plant is optically detected, and the image within the image area actually corresponds to a width of 18 m and a height of 12 m. Thus, the image of this optical detection has a resolution of 0.46 pixels / mm. Based on the known position and location information of the aircraft connected to the captured image, by recognizing characteristic features of the wind power plant such as blade tips, a CAD model of the wind power plant can be generated or the generated model can be corrected. Due to the long distance and low resolution, this step can be carried out with little time and effort. Detecting features can be carried out especially by using methods of machine learning. It should be noted that the distance can also be in a certain area or interval, for example, up to 25 m or up to 20 m, or in the range from 20 m to 25 m. Furthermore, the distance can also be 20 m, for example.

[0087] Thereafter, based on the CAD model, waypoints for an inspection flight, for example for tower inspection and / or blade inspection, can be generated. These waypoints can also be generated again by using AI. Both the evaluation and the generation of waypoints can be carried out after landing after the calibration flight or also during the calibration flight by the aircraft itself. In order to be able to detect defects with sufficient accuracy, for example, in an inspection flight that is subsequently carried out autonomously, the distance of the aircraft to the wind power plant is reduced. For blade inspection, a distance of, for example, 7 m can be set so that the generated image actually corresponds to a width of about 5 m and a height of about 3.3 m, which results in a resolution of 1.63 pixels / mm. In the case of a blade with an exemplary length of 70 m and having three sides and an overlap of 20 cm, 75 images result for each of the 25 images of the sides of the blade. Similarly, for tower inspection, 304 images result at a resolution of 1.28 pixels / mm.

[0088] To evaluate the inspection flight images, AI can be used again. As already mentioned above, AI can itself already classify the images during flight and divide them into images showing damage and images not showing damage. Alternatively, this can also be done on an external computer after landing. A great advantage of the machine learning method is revealed based on a large number of images that would result in a great deal of time and labor if the evaluation were performed by a human. By the idea of the present invention of detecting again individually the areas of the wind power plant containing damage, during the inspection of the tower and / or blades, or for the entire surface and / or blades, the direct time-consuming and data-intensive generation of high-resolution images can be omitted. By the position and location information of the drone that can be connected to each image where damage is detected, each part can be approached again during the defective flight. Alternatively, the detection can also be performed at a second or higher resolution during the inspection of the tower or blades. The distance of the aircraft to the object can be reduced to 3 m, or (for example, during an intermediate landing) respective lenses can be attached, or (for example, during flight) the zoom setting can be adapted as appropriate. Thereby, a resolution of 3.87 pixels / mm can be obtained.

[0089] With such a high resolution, even the smallest damage can be detected and classified. Thus, for example, strict legal requirements regarding the safety of the power plant can be implemented. By reducing the number of detailed images during the new optical detection with the increased or second resolution, the method of the present invention is combined with less time and resource effort by a pre-selection of the areas of the wind power plant to be considered. Moreover, due to the use of multiple autonomously flying drones, each method is highly scalable. A further option for scaling is accuracy, by further reducing the distance and improving the image sensor, both the inspection flight and the defective flight can be improved. Furthermore, even using a zoom lens, more images with a higher resolution can be generated.

[0090] In FIGS. 5 and 6 below, the method of the present invention is briefly summarized.

[0091] FIG. 5 shows a flowchart of a method for detecting features of an object according to an embodiment of the present invention. FIG. 5 shows method steps 510 to 530 in order, but this is merely exemplary. Therefore, the method steps can also be used in a modified order. Step 510 includes flying along the object and optically detecting at least a part of the object at a first resolution by at least the capture unit 130 to generate a plurality of images, each image representing at least a partially different region of the object. Step 520 includes, for example, automatically evaluating a plurality of images for classifying the generated images into images without features and images with features. Step 530 includes optically redetecting the region of the object where the assigned image has features at a second resolution higher than the first resolution.

[0092] FIG. 6 shows a flowchart of a further method for detecting features of an object according to an embodiment of the present invention. FIG. 6 shows method steps 610 to 630 in order, but this is merely exemplary. Therefore, the method steps can be applied in a modified order. Step 610 includes flying along the object and optically detecting at least a part of the object by at least one capture unit to generate a plurality of images, each image representing at least a partially different region of the object. For one region, an image having a first resolution and a plurality of partial images each having a second resolution higher than the first resolution are generated. Step 620 includes, for example, automatically evaluating a plurality of images for classifying the generated images into images without features and images with features. Step 630 includes providing partial images of the region of the object where the assigned image has features.

[0093] It should be noted that the optical detection according to the embodiment can also include detection in the infrared range by, for example, an infrared camera.

[0094] All lists of materials, environmental impacts, electrical properties, and optical properties described herein are to be considered illustrative and not limiting.

[0095] Although some aspects are described in the context of an apparatus, it will be apparent that these aspects also represent corresponding methods of operation, where each block or device of the apparatus corresponds to a respective method step or a feature of a method step. Similarly, aspects described in the context of a method step represent corresponding apparatus blocks or corresponding apparatus details or features. Some or all of the method steps may be performed by (or using) a hardware device such as a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, some or several of the most important method steps may be performed by such a device.

[0096] Depending on the particular implementation requirements, embodiments of the present invention may be implemented in hardware or software. Implementations may use a digital storage medium, such as a floppy disk, a DVD, a Blu-Ray disk, a CD, a ROM, a PROM, an EPROM, an EEPROM, or a FLASH memory, a hard drive, or another magnetic or optical memory, which electronically stores control signals that cooperate with or are capable of cooperating with a programmable computer system so that respective methods are performed. Thus, the digital storage medium may be computer-readable.

[0097] Some embodiments according to the present invention include a data carrier containing electronically readable control signals that cooperate with or are capable of cooperating with a programmable computer system so that one of the methods described herein is performed.

[0098] In general, an embodiment of the present invention may be implemented as a computer program product having program code, the program code being operative to perform one of the methods when the computer program product is executed on a computer.

[0099] The program code may be stored, for example, on a machine-readable carrier.

[0100] Other embodiments include a computer program for performing one of the methods described herein, the computer program being stored on a machine-readable carrier.

[0101] In other words, thus, an embodiment of the method of the present invention is a computer program including program code for performing one of the methods described herein when the computer program is executed on a computer.

[0102] Thus, a further embodiment of the method of the present invention is a data carrier (or digital storage medium or computer-readable medium) having recorded thereon a computer program for performing one of the methods described herein. The data carrier, digital storage medium, or computer-readable medium is typically tangible or non-volatile.

[0103] Thus, a further embodiment of the method of the present invention is a data stream or sequence of signals representing a computer program for performing one of the methods described herein. The data stream or sequence of signals may be configured to be transmitted via, for example, a data communication connection, such as via the Internet.

[0104] A further embodiment comprises processing means, such as a computer or a programmable logic device, configured or adapted to perform one of the methods described herein.

[0105] A further embodiment comprises a computer having installed thereon a computer program for performing one of the methods described herein.

[0106] A further embodiment according to the present invention includes an apparatus or system configured to transmit to a receiver a computer program for performing at least one of the methods described herein. The transmission can be, for example, electronic or optical. The receiver can be, for example, a computer, a mobile device, a memory device, or a similar device. The apparatus or system can include, for example, a file server for transmitting the computer program to the receiver.

[0107] In some embodiments, a programmable logic device (e.g., a field programmable gate array, FPGA) can be used to perform some or all of the functions of the methods described herein. In some embodiments, a field programmable gate array can cooperate with a microprocessor to perform one of the methods described herein. Generally, the methods are preferably performed by any hardware device. This can be general purpose hardware such as a computer processor (CPU), or hardware specific to the method such as an ASIC.

[0108] The apparatus described herein can be implemented, for example, by using a hardware device, or by using a computer, or by using a combination of a hardware device and a computer.

[0109] The apparatus described herein, or any component of the apparatus described herein, can be implemented at least partially in hardware and / or software (a computer program).

[0110] The methods described in this specification can be implemented, for example, by using a hardware device, or by using a computer, or by using a combination of a hardware device and a computer.

[0111] The methods described in this specification, or any component of the methods described in this specification, can be executed at least in part by hardware and / or software.

[0112] The embodiments described above are merely illustrative of the principles of the present invention. It is understood that modifications and variations of the arrangements and details described in this specification will be apparent to those skilled in the art. Accordingly, the present invention is intended to be limited only by the appended claims and not by the specific details presented in the description and explanation of the embodiments herein.

Description of Reference Numerals

[0113] 110 Object 110a Feature 110b Front Surface 120 Unmanned Aerial Vehicle, Aircraft 130 Capture Unit 140 Computer 140a Connection, Communication 140b Communication 140c Classification 150 Flight Trajectory, Trajectory 210 Image 220 Image 220a Image 220b Image 400 Wind Power Plant 410 Tower 420 Pod 430 Rotor 440 Rotor or Blade, Rotor, Blade 450 Rotor Blade or Blade Tip, Blade Tip, Rotor Tip 460 Rotor Blade Flange 470 Partial Image

Claims

1. A method (500) for detecting damage (110a) to an object (110, 400), comprising: (a) optically detecting at least a part (110b) of the object by at least one capture unit (130) while flying along the object (110, 400) to generate a plurality of images (B1 - B4) at a first resolution, each image representing at least a partially different region of the object (110, 400), step (510); (b) evaluating the plurality of images to classify the generated images into an image (210) without the damage and an image (220) with the damage, step (520); and (c) optically detecting again at a second resolution, higher than the first resolution, the region of the object where the assigned image contains the damage (110a), step (530). The method (500).

2. Step (b) is executed after step (510) of flying along the object (110, 400), Step (c) includes the step of approaching the region of the object (110, 400) where the assigned image contains the damage (110a). The method (500) according to claim 1.

3. In step (a) and step (c), the capture unit generates one image each having the same focal length. The method (500) according to claim 2. In step (a), the object (110, 400) is approached such that when the first capture unit (130) generates an image (B1 to B4), the object has a first distance (d １ ) from the first capture unit (130). In step (c), the object is approached such that when the capture unit generates an image (B21, 470), the object has a second distance (d ２ ) that is shorter than the first distance with respect to the object.

4. In step (a) and step (c), the object (110, 400) is approached so as to have the same or a similar distance to the object when the capture unit (130) generates an image, In step (a), the capture unit (130) generates an image having a first focal length, In step (c), the capture unit (130) generates an image (B21, 470) having a second focal length longer than the first focal length. The method (500) according to claim 2.

5. In step (a), the capture unit (130) uses at least one of a first camera, a first lens, and a first camera lens having the first focal length, or a zoom lens having a first zoom lens setting corresponding to the first focal length. ​ Step (c) includes replacing the first camera of the capture unit (130) with a second camera having the second focal length, and / or replacing the first lens of the capture unit (130) with a second lens having the second focal length, and / or replacing the first camera lens of the capture unit (130) with a second camera lens having the second focal length, or setting the zoom lens of the capture unit to a second zoom setting corresponding to the second focal length. The method (500) according to claim 4.

6. The method (500) according to claim 2, wherein the step of optically detecting the region again in step (c) includes generating a plurality of partial images (B21 to B24) of the region, each having the second resolution.

7. The position and / or location information of the capture unit (130) is assigned to each image generated in step (a). In step (c), the region of the object (110, 400) to be flown along is determined by using the position and / or location information of the image (220) including the damage (110a). The method (500) according to any one of claims 2 to 6.

8. An unmanned aircraft (120), such as a drone, equipped with the capture unit (130) flies along the object (110, 400). Step (b) includes transmitting the images (B1 to B4) generated in step (a) from the unmanned aircraft to a computer (140), such as a laptop computer, and evaluating the images by the computer (140). The step of evaluating the images includes evaluating the images in an automated manner. The method (500) according to any one of claims 2 to 7.

9. In step (a), the unmanned aircraft (120) autonomously flies along the object (110, 400). Step (b) includes generating a waypoint (WP2) by using the position and / or location information of the image (220) including the damage, and transmitting the waypoint to the unmanned aircraft (120). In step (c), the unmanned aircraft (120) autonomously approaches the region of the object by using the waypoint (WP2). The method (500) according to claim 8.

10. An unmanned aircraft (120) equipped with the capture unit (130) flies autonomously along the object. The unmanned aircraft (120) includes a computer (140), and step (b) includes the step of evaluating the image by the computer (140) of the unmanned aircraft (120) and generating a waypoint (WP2) by using the position and / or location information of the image (220) including the damage. The step of evaluating the image includes the step of evaluating the image in an automated method. In step (c), the unmanned aircraft (120) autonomously approaches the region of the object (110, 400) by using the waypoint (WP2). The method (500) according to any one of claims 2 to 7.

11. The step (510) of flying along the object (110b) in step (a) is a step of autonomously flying along the object by the unmanned aircraft (120), and the unmanned aircraft includes the at least one capture unit (130). Step (b) includes the step of generating a waypoint by using the position and / or location information of the image including the damage, and the step of transmitting the waypoint to the unmanned aircraft. Step (c) includes the step of autonomously flying along the region of the object by using the waypoint. The method according to any one of claims 1 to 10.

12. In step (a), images (B1 to B4) of a region are generated. In step (b), the images generated in step (a) are classified for a further region before generating the images. When the image (B2) is classified as including the damage (110a) in step (b), before images (B3, B4) are generated for the further region, in step (c), before generating further images, the region is optically detected again. In step (b), when the images (B1, B3, B4) are classified as not including the damage (110a), an image is generated for the further region. The method (500) according to claim 1, wherein steps (a) to (c) are executed while flying along the object (110, 400).

13. In steps (a) and (c), the capture unit (130) generates one image each having the same focal length. In step (a), the object (110, 400) is approached such that when the capture unit (130) generates an image (B2), the object (110, 400) has a first distance (d １ ) from it. In step (c), the distance of the capture unit (130) with respect to the object (110, 400) is shortened to a second distance (d ２ ) shorter than the first distance. The method (500) according to claim 12.

14. In step (a), the object (110, 400) is approached so as to have a first distance with respect to the object (110, 400) when the capture unit (130) generates an image (B2), and the capture unit (130) generates an image (B2) having a first focal length. In step (c), the distance of the capture unit (130) with respect to the object (110, 400) is the same as or similar to the first distance, and the capture unit (130) generates an image (B21, 470) having a second focal length longer than the first focal length. The method (500) according to claim 12.

15. The capture unit (130) includes at least one of a plurality of lenses, a zoom lens, a plurality of cameras, and a plurality of camera lenses. In step (a), the capture unit (130) uses a first camera and / or a first lens and / or a first camera lens having the first focal length, and / or sets the zoom lens to a first zoom setting according to the first focal length. In step (c), the capture unit (130) uses a second camera and / or a second lens and / or a second camera lens having the second focal length, and / or sets the zoom lens to a second zoom setting according to the second focal length. The method (500) according to claim 14.

16. The method (500) according to claim 12, wherein the step of optically redetecting the region in step (c) includes the step of generating a plurality of partial images (B21 to B24) of the region each having the second resolution.

17. The unmanned aircraft (120) including the capture unit (130) autonomously flies along the object (110, 400), the unmanned aircraft (120) includes a computer (140), and step (b) is a step of evaluating the image by the computer (140) of the unmanned aircraft, the step of evaluating the image includes the step of evaluating the image in an automated method, The method (500) according to any one of claims 12 to 16.

18. (d) For example, in order to classify or catalog the detected damage (110a), a step of transmitting the image generated in step (c) to an evaluation unit (140) The method (500) according to any one of claims 1 to 17, including

19. A method (600) for detecting damage (110a) to an object (110, 400), the method comprising: (a) A step of optically detecting at least a part (110b) of the object by at least one capture unit (130) while flying along the object (110, 400) to generate a plurality of images (B1 to B4), each image representing at least a partially different region of the object, and for one region, an image having a first resolution and a plurality of sub-images (B11 to B44) each having a second resolution higher than the first resolution are generated, step (610); (b) A step of evaluating the plurality of images to classify the generated images into an image (210) not including the damage (110a) and an image (220) including the damage (110a), step (620); (c) A step (630) of providing a sub-image (B21, 470) of a region of the object including the damage (110a) for the assigned image (B2, 220a). including Method (600).

20. In step (a), the object (110, 400) is approached so as to have a certain distance from the object (110, 400) when the capture unit (130) generates the images (B1 to B4) and the sub-images (B11 to B44). In step (a), the capture unit (130) generates the image having a first focal length and the sub-image having a second focal length longer than the first focal length. The method (600) according to claim 19.

21. The method (600) according to claim 20, wherein the capture unit (130) comprises a zoom lens that uses a first zoom setting corresponding to the first focal length when generating the images (B1 to B4) and uses a second zoom setting corresponding to the second focal length when generating the partial images (B11 to B44).

22. An unmanned aerial vehicle (120), such as a drone, comprising the capture unit (130), autonomously flies along the object (110, 400), Step (b) includes transmitting the images (B1 to B4) and the partial images (B11 to B44) generated in step (a) from the unmanned aerial vehicle (120) to a computer (140), such as a laptop computer, and evaluating the images by the computer (140). The step of evaluating the images includes evaluating the images in an automated method. Step (c) includes providing, by the computer (140), the partial images of the regions assigned to the images. The method (600) according to any one of claims 19 to 21.

23. An unmanned aerial vehicle (120), such as a drone, comprising the capture unit (130), autonomously flies along the object (110, 400), The unmanned aerial vehicle (120) comprises a computer (140), and step (b) includes evaluating the images (B1 to B4) and the partial images (B11 to B44) by the computer (140) of the unmanned aerial vehicle (120). The step of evaluating the images (B1 to B4) and the partial images (B11 to B44) includes evaluating the images (B1 to B4) and the partial images (B11 to B44) in an automated method. In step (c), the unmanned aerial vehicle (120) transmits the partial images to an evaluation unit (140) to classify or catalog, for example, the detected damage (110a). The method (600) according to any one of claims 19 to 21.

24. The method (500, 600) according to any one of claims 1 to 23, wherein step (b) includes AI or machine learning.

25. The method (500, 600) according to any one of claims 1 to 24, wherein the object (110, 400) includes an energy generation plant, such as a wind power plant (400) or a solar power plant, or an industrial plant, such as an oil platform, a factory plant, a refinery, or a building, such as a multi-story building, or an infrastructure means.

26. A drone (120), for example, a drone for detecting damage (110a) of an object (110, 400), comprising at least one capture unit (130) for generating an image by optical detection, wherein the drone (120) - flies along the object (110, 400) and optically detects at least a part (110b) of the object by the capture unit (130) at a first resolution to generate a plurality of images (B1 to B4), each image representing at least a partially different region of the object (110, 400), - re-optically detects the region of the object (110, 400) including the damage (110a) of the assigned image (B2) at a second resolution higher than the first resolution and can be controlled to perform, the drone comprises a computer configured to evaluate the plurality of images to classify the generated images into the images (210) not including the damage (110a) and the images (220) including the damage (110a). Drone (120).

27. The drone (120) can be controlled to fly autonomously along the object and fly autonomously along the region of the object (110, 400) where the assigned image (B2) includes the damage (110a) using waypoints. The drone (120) is configured to receive waypoints, and the waypoints are generated by using the position and / or location information of the images including the damage. The drone according to claim 26.

28. A system for detecting damage (110a) of an object (110, 400), comprising a drone (120), for example, a drone. The unmanned aircraft (120) can be controlled to fly along the object (110, 400) in order to optically detect at least a part (110b) of the object at a first resolution by means of at least one capture unit (130) for generating a plurality of images (B1 to B4), each image representing at least partially different regions of the object (110, 400). The system is configured to evaluate the plurality of images in order to classify the generated images into an image (210) without the damage (110a) and an image (220) with the damage (110a). The unmanned aircraft (120) can be controlled to optically detect again at a second resolution, which is higher than the first resolution, the region of the object where the assigned image (B2) includes the damage (110a). System.

29. The unmanned aircraft comprises the at least one capture unit, and the unmanned aircraft flies autonomously along the object and can be controlled to autonomously approach, by using waypoints, the region of the object (110, 400) where the assigned image (B2) includes the damage (110a). The unmanned aircraft (120) is configured to transmit the plurality of images to a computer. The system comprises a computer. The computer is configured to at least partially evaluate the plurality of images in an automated method in order to classify the generated images into an image (210) without the damage (110a) and an image (220) with the damage (110a). The computer is configured to generate waypoints by using the position and / or location information of the image with the damage. The computer is configured to transmit the waypoints to the unmanned aircraft (120). The system according to claim 28.

30. A system for detecting damage (110a) to an object (110, 400), comprising an unmanned aircraft (120), such as a drone, wherein the unmanned aircraft (120) ​ - Flying along the object (110, 400) and optically detecting at least a part (110b) of the object by a capture unit (130) to generate a plurality of images (B1 to B4), each image representing at least partially different regions of the object (110, 400). - For each region, generating an image (B1 to B4) having a first resolution and a plurality of partial images (B11 to B44) each having a second resolution higher than the first resolution. It can be controlled to perform. The system is evaluating the plurality of images to classify the generated images into an image (210) without the damage (110a) and an image (220) with the damage (110a). For example, to classify or catalog the detected damage, providing a partial image (B21) of the region of the object (110, 400) where the assigned image contains the damage (110a). configured as a system.

31. A method (500) for detecting damage (110a) of an object (110, 400), wherein the damage of the object is the damage of the object, and the method includes (a) flying along the object (110, 400) and optically detecting at least a part (110b) of the object at a first resolution by at least one capture unit (130) to generate a plurality of images (B1 to B4), each image representing at least partially different regions of the object (110, 400), step (510); (b) evaluating the plurality of images to classify the generated images into an image (210) without the damage and an image (220) with the damage, step (520); (c) optically detecting again at a second resolution higher than the first resolution the region of the object where the assigned image contains the damage (110a), step (530). including In step (a), images (B1 to B4) of regions are generated. In step (b), the images generated in step (a) are classified for further regions before generating the images. If, in step (b), the image (B2) is classified as including the damage (110a), then before images (B3, B4) are generated for the further area and before further images are generated, in step (c), the area is optically detected again, If, in step (b), the images (B1, B3, B4) are classified as not including the damage (110a), then an image is generated for the further area Steps (a) to (c) are executed while flying along the object (110, 400), In step (a), the object (110, 400) is approached such that the capture unit (130) has a first distance to the object (110, 400) when the capture unit (130) generates an image (B2), and the capture unit (130) generates an image (B2) having a first focal length, In step (c), the distance of the capture unit (130) to the object (110, 400) is the same as or similar to the first distance, and the capture unit (130) generates an image (B21, 470) having a second focal length that is longer than the first focal length Method (500). **Claim 32**: A method (500) for detecting damage (110a) of an object (110, 400), (a) flying along the object (110, 400) and optically detecting at least a part (110b) of the object at a first resolution by at least one capture unit (130) to generate a plurality of images (B1 - B4), each image representing at least a partially different area of the object (110, 400), step (510); (b) evaluating the plurality of images to classify the generated images into an image (210) not including the damage and an image (220) including the damage, step (530); (c) optically detecting again at a second resolution, which is higher than the first resolution, an area of the object including the damage (110a) for which the assigned image includes the damage (110a), step (530), Step (b) is executed after step (510) of flying along the object (110, 400), Step (c) includes approaching an area of the object (110, 400) for which the assigned image includes the damage (110a), The position and / or location information of the capture unit (130) is assigned to each image generated in step (a), In step (c), the region of the object (110, 400) to be flown along is determined by using the position and / or location information of the image (220) including the damage (110a), Method (500). **Claim 33**: A method (500) for detecting damage (110a) to an object (110, 400), (a) optically detecting at least a part (110b) of the object by at least one capture unit (130) while flying along the object (110, 400) to generate a plurality of images (B1 to B4) at a first resolution, wherein each image represents at least a partially different region of the object (110, 400), step (510); (b) evaluating the plurality of images to classify the generated images into an image (210) not including the damage and an image (220) including the damage, step (520); (c) optically detecting again at a second resolution higher than the first resolution the region of the object where the assigned image includes the damage (110a), step (530), Step (b) is executed after step (510) of flying along the object (110, 400), Step (c) includes the step of approaching the region of the object (110, 400) where the assigned image includes the damage (110a), An unmanned aerial vehicle (120) equipped with the capture unit (130), such as a drone, autonomously flies along the object (110, 400), Step (b) includes the step of transmitting the images (B1 to B4) generated in step (a) from the unmanned aerial vehicle to a computer (140), such as a laptop computer, and the step of evaluating the images by the computer (140), The step of evaluating the images includes the step of evaluating the images in an automated method, Step (b) includes the step of generating a waypoint (WP2) by using the position and / or location information of the image (220) including the damage and the step of transmitting the waypoint to the unmanned aerial vehicle (120), or The unmanned aircraft (120) is provided with a computer (140), and step (b) includes the computer (140) of the unmanned aircraft (120) evaluating the image and generating a waypoint (WP2) by using the position and / or location information of the image (220) including the damage, and the step of evaluating the image includes the step of evaluating the image in an automated manner, In step (c), the unmanned aircraft (120) autonomously approaches the area of the object by using the waypoint (WP2). Method (500). **Claim 34**: A method (600) for detecting damage (110a) to an object (110, 400), the method comprising: (a) flying along the object (110, 400) and optically detecting at least a part (110b) of the object by at least one capture unit (130) to generate a plurality of images (B1 - B4), each image representing at least partially different areas of the object, and for one area, an image having a first resolution and a plurality of sub-images (B11 - B44) each having a second resolution higher than the first resolution are generated, step (610); (b) evaluating the plurality of images to classify the generated images into an image (210) not including the damage (110a) and an image (220) including the damage (110a), step (620); (c) providing a sub-image (B21, 470) of the area of the object where the assigned image (B2, 220a) includes the damage (110a), step (630). and including An unmanned aircraft (120), such as a drone, provided with the capture unit (130) flies autonomously along the object (110, 400). Step (b) includes the steps of transmitting the images (B1 to B4) and partial images (B11 to B44) generated in step (a) from the unmanned aircraft (120) to a computer (140), such as a laptop computer, and evaluating the images by the computer (140), the step of evaluating the images including evaluating the images by an automated method, step (c) including the step of providing, by the computer (140), the partial images of the regions assigned to the images, or, the unmanned aircraft (120) includes a computer (140), step (b) including the step of evaluating the images (B1 to B4) and the partial images (B11 to B44) by the computer (140) of the unmanned aircraft (120), including the step of evaluating the images (B1 to B4) and the partial images (B11 to B44) in an automated method, the step of evaluating the images (B1 to B4) and the partial images (B11 to B44) including the step of evaluating the images (B1 to B4) and the partial images (B11 to B44) in an automated method, in step (c), the unmanned aircraft (120) transmitting the partial images to an evaluation unit (140), for example, to classify or catalog the detected damage (110a), Method (600).

Images