Method and device for setting a receiving unit for carrying out an inspection of an object

Abstract

Exemplary embodiments according to the present invention comprise a method for setting a receiving unit (110) for carrying out an inspection of an object (120, 121, 122, 123, 223, 320), said method comprising the following steps: providing an optical reference receptacle (131, 430) which is created by means of the receiving unit and which at least partially contains the object to be inspected and contains surroundings (140, 240, 340) of the object; identifying the object within the optical reference receptacle; determining brightness information (160) based on at least a region of the reference receptacle which has at least a partial region of the object; determining an inspection exposure setting for the receiving unit using the brightness information. The invention also relates to corresponding devices.

Description

[0001] Method and device for adjusting a recording unit for carrying out an inspection of an object

[0002] Description

[0003] Technical area

[0004] Embodiments include methods and apparatus for adjusting a recording unit for performing an inspection of an object.

[0005] Furthermore, embodiments include methods and apparatus for object inspection using object-based brightness histograms.

[0006] Background of the invention

[0007] Inspection and maintenance of systems is a key component for safe and efficient operation. Inspections, especially for very large and / or difficult-to-access objects or system components, present several challenges.

[0008] Large turbines are usually located outdoors, meaning that visual inspections, in particular, are affected by the sometimes daily changing lighting conditions, e.g., due to changes in weather conditions. In some cases, it may not be possible to postpone a corresponding inspection indefinitely for safety or economic reasons, or to accept the downtime required for inspection of such a turbine multiple times, for example, when lighting conditions are unfavorable. Furthermore, such turbines, such as wind turbine blades, often have to be inspected from multiple angles, meaning that unfavorable backlighting situations, for example, are unavoidable.

[0009] Therefore, there is a need for a concept for the inspection of an object that allows the inspection to be carried out as independently as possible from the prevailing lighting conditions and thereby achieves an improved compromise between robustness and accuracy and complexity and effort of the inspection.

[0010] Such a concept is provided by the subject matter of the independent patent claims. Further developments according to the invention are defined in the subclaims.

[0011] Summary of the Invention Embodiments according to the present invention comprise a method for setting a recording unit for carrying out an inspection of an object, comprising the following steps: providing an optical reference image generated by the recording unit, which at least partially contains the object to be inspected and an area surrounding the object; detecting the object within the optical reference image; determining brightness information based on at least one area of ​​the reference image, which area includes at least a partial area of ​​the object; and determining an inspection exposure setting for the recording unit using the brightness information.

[0012] The inventors have recognized that, for the purposes of an object inspection, adjusting an inspection exposure setting using brightness information based on an area of ​​the reference image that includes the object or at least a part thereof enables more meaningful inspection results, and furthermore, a corresponding inspection method performed using the inspection exposure setting is more robust with regard to environmental influences on the lighting conditions.

[0013] By setting exposure parameters, such as aperture or exposure time, based on the object's pixels in the reference image, the exposure can be adjusted so that the parts of an inspection image relevant for inspection, generated using the inspection exposure setting, exhibit sufficient brightness for the object features to be inspected. This is made possible by detecting the object within the optical reference image.

[0014] In particular, this prevents over- or underexposure of the object being inspected due to particularly bright or dark parts of the image, such as the background of the object. This allows for good inspection results even in difficult backlight situations.

[0015] According to one embodiment, the object has an optically detectable object characteristic, such as a color, a pattern and / or a color pattern, and determining the inspection exposure setting for the recording unit is performed using the brightness information and the object characteristic.

[0016] The object characteristic can, for example, be an object-specific, a priori known property (e.g., based on prior knowledge of a system to be inspected), which can form an input variable (e.g., for manual input) of a corresponding method according to the invention. For example, such an object characteristic can also be stored in a program according to the invention, e.g., a computer program. For example, it can be taken into account that a blade of a wind turbine is red (e.g., = dominant color channel of the object in the reference image), or at least predominantly red, so that the evaluation of the brightness is focused on the red color component in the image, e.g., carried out with emphasis. For example, a histogram associated with the reference image for a corresponding color channel, e.g.,red, with regard to exposure settings, can be optimized so that a recording of the same scene, under the same or similar conditions, has a brightness of the corresponding color channel that is sufficient for inspection purposes.

[0017] This allows for improvements or optimization of exposure settings to be adapted to individual object properties in order to enable meaningful and robust inspections.

[0018] According to one embodiment, the method further comprises recognizing the optically detectable object characteristic based on the optical reference image.

[0019] The method can thus also be automated with regard to the recognition of the object characteristics, whereby, for example, an algorithm, e.g. a CL, can recognize an object, such as a blade of a wind turbine, and can determine the object characteristics based on the position in the image by means of which or with the aid of which the inspection exposure setting can be determined.

[0020] According to one exemplary embodiment, the brightness information comprises information about a brightness distribution, e.g. a histogram, of the at least one region of the reference image, and the method further comprises determining at least one brightness threshold based on the object characteristic and determining the inspection exposure setting for the recording unit using the brightness information and the at least one brightness threshold, such that an inspection image of the object, which is generated by means of the recording unit with the inspection exposure setting under an identical or at least similar illumination situation of the object as in the reference image, has a brightness distribution which satisfies the brightness threshold. It should be noted here that, in particular, a plurality of brightness thresholds can also be used, e.g. in the sense of upper and lower limits, e.g.each specific to a particular object characteristic.

[0021] Thus, a brightness threshold can be set individually for each object, based on the respective object characteristics, on the basis of which, for example, from preliminary tests or based on information about defects or damage to be detected, a meaningful inspection result can be achieved with sufficient certainty on the basis of an inspection image generated using the inspection exposure setting.

[0022] According to one embodiment, the method is a method for adjusting a recording unit for carrying out an inspection of the object with respect to an object feature to be optically recognized, and the determination of the at least one brightness threshold based on the object feature to be recognized is carried out such that the object feature to be recognized is recognizable from the inspection image of the object.

[0023] This allows a corresponding object feature, e.g., a characteristic location on the object, such as a blade tip or rotor blade hub of a wind turbine, or damage, such as a crack or hole, to be reliably detected. Based on the object feature to be detected, the inspection exposure setting can be optimized, or the detectability of the object feature can form a corresponding optimization criterion, e.g., for generating or selecting an appropriate brightness threshold.

[0024] According to one embodiment, the object feature is damage and / or potential damage to the object. The exposure adjustment according to the invention has advantages, particularly for safety-critical applications, such as the detection of defects, due to a robust inspection based on it that is largely independent of environmental influences.

[0025] According to one embodiment, the optically detectable object characteristic is a color, a color pattern, and / or a color distribution of the object. The optical reference image has multiple color channels. The brightness information is color-specific information about a brightness distribution. Determining the at least one brightness threshold involves determining one or more color-specific brightness thresholds. The inventors have recognized that color-channel-specific consideration of brightness information for determining the inspection exposure setting can lead to better inspection results.

[0026] According to one embodiment, all pixels of the object in the optical reference recording and / or a contiguous subset of pixels of the object in the optical reference recording and / or a predetermined pattern of pixels of the object in the optical reference recording are used to determine the brightness information.

[0027] The brightness information can be evaluated, for example, in the form of a histogram. Furthermore, the inventors have recognized that, depending on the object and / or feature to be detected, the inspection exposure setting can be determined during the inspection or, as an intermediate step, a brightness threshold can be determined, for example, based on all object points or only a selected subset. By cleverly selecting the points to be evaluated for the specific application, computing power can be saved, for example in particular if the corresponding evaluation is carried out on-board a drone, which can, for example, take reference and inspection images in one takeoff. Furthermore, in some applications, the significance of the brightness information with regard to optimizing an inspection can be improved if not all points of the object are taken into account, e.g.not those points which represent areas of no interest for inspection.

[0028] According to one embodiment, the optical reference image is based on a reference exposure setting of the recording unit, and the determination of the inspection exposure setting for the recording unit is carried out using the brightness information and at least one brightness threshold. Furthermore, the method comprises, as optional features, a comparison of the brightness information with the at least one brightness threshold, wherein, if the brightness information satisfies the at least one brightness threshold, the reference exposure setting is determined as the inspection exposure setting, and wherein, if the brightness information does not satisfy the at least one brightness threshold, the steps

[0029] Determining an adjusted reference exposure setting using the comparison; Generating an adjusted reference exposure to simulate generating the reference exposure using the adjusted reference exposure setting;

[0030] Determining an adjusted brightness information based on at least one area of ​​the adjusted reference image, which has at least a partial area of ​​the object; and

[0031] Comparing the adjusted brightness information with the at least one brightness threshold; carried out iteratively until the adjusted brightness information satisfies the at least one brightness threshold, wherein the adjusted reference exposure setting by means of which the brightness threshold is satisfied is then determined as the inspection exposure setting.

[0032] Thus, the exposure setting can be adjusted iteratively, for example, based on a simulated image of the scene of the adjusted reference image (i.e., the reference shot). Furthermore, the adjusted reference shot can also be based on a new image of the object, and in particular the scene of the original reference shot, using the recording unit, i.e., in other words, a recording device.

[0033] According to one embodiment, the object is a first object, the optical reference image has a second object, and the method further comprises detecting the second object within the optical reference image, determining second brightness information based on at least one region of the reference image which has at least a partial region of the second object, determining a second inspection exposure setting for the recording unit using the second brightness information, generating a first optical inspection image of the first object by means of the recording unit with the inspection exposure setting, and generating a second optical inspection image of the second object by means of the recording unit with the second inspection exposure setting.

[0034] This means that multiple objects can each be scanned with improved or optimized exposure settings to achieve a meaningful inspection. Such an adjustment can therefore be carried out, for example, starting from a single image of the two objects. The two objects can then each be scanned in separate images with individual exposure settings. This means that even if there are a large number of objects to be inspected, a separate reference image does not have to be created for each object. At this point, however, it should be explicitly pointed out that the previous explanations with regard to two objects are to be understood as examples. According to exemplary embodiments, any number of objects can be detected and, for example, correspondingly adjusted exposure settings can be determined for each.

[0035] According to one embodiment, the method further comprises fusing the first and second optical inspection images.

[0036] For example, the first object can be cut out from the first optical inspection image and / or the second object can be cut out from the second optical inspection image and inserted into a common image (e.g. into the reference image, e.g. into the first inspection image, e.g. into the second inspection image) so that both objects are each imaged with sufficient brightness for an inspection within the fusion image. A wide variety of fusion options are possible; embodiments are not limited to the above-described cutting out. For example, the two inspection images can also be superimposed in a weighted manner. The fusion image thus enables improved inspection of both objects.

[0037] According to one embodiment, the object is a first object, the optical reference image has a second object, and the method further comprises detecting the second object within the optical reference image; determining common brightness information based on the at least one area of ​​the reference image that has at least a partial area of ​​the first object, and based on at least one area of ​​the reference image that has at least a partial area of ​​the second object; determining a common inspection exposure setting for the image acquisition unit using the common brightness information;or determining a second item of brightness information based on at least one area of ​​the reference image which has at least a partial area of ​​the second object and determining a common inspection exposure setting for the recording unit using the first and second items of brightness information and generating an optical inspection image of the first and second object by means of the recording unit with the common inspection exposure setting.;

[0038] According to embodiments, brightness information can thus be used that depicts the brightness of both the first (or parts of the first) and the second (or parts of the second) object to determine an exposure setting based thereon, which can be used for both objects, e.g., for the purposes of a meaningful inspection. For example, as explained above, a common piece of brightness information, such as a common histogram, can be evaluated, or two individual pieces of brightness information, such as histograms, each depicting the brightness of one or the other object.

[0039] The common inspection exposure setting can therefore represent a sufficiently good compromise for an inspection in order to inspect a large number of objects with just a few image captures.

[0040] According to one embodiment, the recording unit is a movable (i.e., e.g., mobile) recording unit, and the method further comprises moving the recording unit toward the object (e.g., flying toward the object, e.g., with a recording unit in the form of a drone, e.g., approaching the recording unit to the object, e.g., moving the recording unit toward the object; e.g., moving the recording unit to a predetermined position with respect to the object (e.g., relative to the object)); generating the optical reference recording; and generating an inspection recording of the object by means of the recording unit with the inspection exposure setting.

[0041] The reference image and the inspection image can be created, for example, during separate approaches or within a single flight. This allows the evaluation and determination of the improved exposure parameterization to be performed, for example, during a stopover using an external computer, or on-board during the flight, or via data transfer to a corresponding external computer.

[0042] It should also be noted that according to embodiments the movable recording unit can be designed, for example, in the form of a portable, e.g. static or e.g. mobile, system or also, for example, another movable carrier system or carrier, or can be part of such a system or can be moved by means of such a system. According to one embodiment the method further comprises generating an initial image of the object; detecting the object in the initial image and evaluating the initial image; determining a trigger point for the recording unit for generating the reference image based on the evaluation of the initial image, such that the object is in a predetermined image area of ​​the reference image; and positioning the recording unit at the trigger point (e.g. flying towards the trigger point) and generating the optical reference image of the object at the trigger point using the recording unit.

[0043] This ensures, for example, that relevant object parts are located in a central image section, thus avoiding distortion in peripheral areas. Furthermore, the image area can be defined in such a way that a feature, such as damage, that is to be inspected can be identified not only by the correct brightness setting but also by the resulting resolution.

[0044] According to one embodiment, the detection of the object within the optical reference image and / or the determination of the inspection exposure setting for the imaging unit is performed automatically, e.g., based on artificial intelligence. Thus, reliable and rapid detection can be achieved.

[0045] According to one embodiment, the detection of the object within the optical reference image is performed by means of a first neural network and the determination of the inspection exposure setting for the image acquisition unit is performed using the brightness information by means of a second neural network.

[0046] The inventors have recognized that a two-part use of artificial intelligence enables an improvement in the achievable exposure settings, so that the first network can be set (or trained) for object detection and the second network for optimizing the exposure.

[0047] It should be clearly noted that two neural networks (NN) do not necessarily, for example, always have to be used; it can also be just one (or, for example, no) neural network, or there can be multiple neural networks. For example, it may only be crucial that the necessary steps can be decided. It should be noted here that, according to embodiments, a combination of computer vision (CV) (computer-based vision; computer vision) and neural networks (NN) can also be used. For example, according to embodiments, object recognition can be carried out using CV and / or brightness information can (e.g., likewise) be determined using CV.

[0048] Thus, embodiments include any combination (e.g., object recognition + determining the inspection exposure setting) of computer vision and machine learning approaches (machine learning, e.g., machine vision), e.g., neural networks, e.g., CV+NN, CV+CV, NN+CV, NN+NN, or even just CV and just NN.

[0049] According to one embodiment, the object is a wind turbine or a part of a wind turbine, such as a rotor blade, a rotor blade hub, a rotor blade flange, a tower and / or a nacelle.

[0050] The inventors recognized that the problem of over- or underexposure occurs particularly with wind turbines, especially with large and often narrow turbines (high tower heights with relatively small tower diameters, e.g., compared to a cooling tower of a power plant). Large portions of the background in the form of sky, horizon, and / or surrounding landscape can be visible in a corresponding inspection image, and thus, backlighting situations can occur. Furthermore, such turbines often do not stand out strongly from the background, e.g., a white turbine against a bright sky, so that inspections can be significantly improved using the inventive approaches.

[0051] According to one embodiment, the method is a method for inspecting a component of a wind turbine with respect to optically detectable damage and / or potential damage, and the recording unit is a drone. Furthermore, the method includes, as optional features, flying toward the component of the wind turbine using the recording unit; generating an optical reference image of the component of the wind turbine using the recording unit; automated detection of the component of the wind turbine within the optical reference image; determining a brightness distribution based on a subset of sample points of the reference image that includes the detected component of the wind turbine;and determining an inspection exposure setting for the recording unit using the brightness distribution, so that damage and / or potential damage to the component of the wind turbine can be optically detected in an inspection image of the component of the wind turbine, which is generated by the recording unit with the inspection exposure setting under an identical or at least similar illumination situation of the component of the wind turbine as in the reference image.;

[0052] Embodiments according to the present invention further comprise a device for adjusting a recording unit for carrying out an inspection of an object, wherein the device has an evaluation unit which is designed to obtain an optical reference recording generated by means of a recording unit, which at least partially contains the object to be inspected and an environment of the object; to recognize the object within the optical reference recording; to determine brightness information based on at least one region of the reference recording, which has at least a partial region of the object; and to determine an inspection exposure setting for the recording unit using the brightness information.

[0053] According to one embodiment, the device comprises the recording unit, which is designed to generate the optical reference recording.

[0054] According to one embodiment, the recording unit is a drone for flying over the object, and the recording unit is configured to take the reference image and an inspection image of the object using the inspection exposure setting without an intermediate landing. Thus, the necessary time expenditure can be kept to a minimum.

[0055] According to one embodiment, the recording unit comprises the evaluation unit or the device comprises a communication unit which is designed to provide the evaluation unit with information about the optical reference recording from the recording unit and to provide the recording unit with information about the inspection exposure setting from the evaluation unit.

[0056] The previously described devices can be based on the same considerations as the methods described above. A device according to an embodiment can have all the features, functionalities, and details that are also described with reference to a method according to the invention, or can be supplemented by them, both individually and in combination. The same applies accordingly to methods according to the invention with regard to features, functionalities, and details that are disclosed with reference to a device according to the invention.

[0057] Embodiments according to the present invention are explained in more detail below with reference to the accompanying figures. With regard to the schematic figures shown, it should be noted that the illustrated functional blocks are to be understood both as elements or features of the device according to the disclosure and as corresponding method steps of the method according to the disclosure, and corresponding method steps of the method according to the disclosure can also be derived therefrom. They show:

[0058] Fig. 1 is a schematic view of an object detection and schematic representations of optical recordings according to an embodiment of the invention;

[0059] Fig. 2 is a photograph of a rotor blade against a background in the form of the sky as an example of a reference photograph according to embodiments of the invention;

[0060] Fig. 3 shows schematic views of an object and a background of the object, Fig. 3a), a section of the view from a) in Fig. 3b), as well as schematic plots of brightnesses of color channels for the view from a) in Fig. 3c) and for the view from b) in Fig. 3d), according to embodiments of the invention; and

[0061] Fig. 4 is a schematic view of a detection of multiple objects and schematic representations of optical recordings according to embodiments of the invention.

[0062] Detailed description of the embodiments according to the figures

[0063] Before exemplary embodiments of the present invention are explained in more detail below with reference to the drawings, it is pointed out that identical, functionally equivalent or equivalent elements, objects and / or structures in the different figures are provided with the same or similar reference numerals, so that the description of these elements shown in different exemplary embodiments is interchangeable or can be applied to one another.

[0064] Fig. 1 shows a schematic view of object detection and schematic representations of optical images according to an embodiment of the invention. Fig. 1 shows a recording unit 110, for example in the form of a drone, and an object 120, for example in the form of a wind turbine with a tower 121, a rotor blade hub 122, and rotor blades 123. The recording unit 110 is designed to generate an image 130 of the object 120. In the example of Fig. 1, the recording unit 110 generates an image of a rotor blade 123.

[0065] A method according to the invention now comprises providing an optical reference image 131 generated by means of the recording unit 110, which contains the object 110 to be inspected at least partially (representation 123' of the wing 123) and an environment 140 (or a representation of the environment) of the object.

[0066] As an example of a recording situation, Fig. 1 shows a constellation in which the recording unit 110 is confronted with backlight 150. Therefore, due to an unfavorable exposure situation, the object, or the representation 123' of the object part, in the reference recording 131 may not stand out clearly from the surroundings 140, e.g., a bright horizon, so that a subsequent inspection can only be carried out with difficulty or not at all.

[0067] The method according to the invention now comprises detecting the object (hereinafter referred to simply as the representation 123' of the object) within the optical reference image 131 and determining brightness information 160, here as an example a histogram (see also, for example, Fig. 3 below), based on at least one region of the reference image which has at least a partial region of the object, i.e. here the rotor blade 123.

[0068] This can be done, for example, with an evaluation unit included in the recording unit, or by communication to an external evaluation unit.

[0069] Based on the brightness information 160, an inspection exposure setting for the recording unit 110 can then be determined. Using the inspection exposure setting, an inspection image 132 can then be generated, for example, in which the exposure is set such that the object or the object part (i.e., the representation 123" of the object part 123) stands out sufficiently clearly from the surroundings 140 to enable a meaningful inspection (e.g., to also detect damage to the wing edges).

[0070] In other words, the method optionally includes moving the recording unit 110 toward the object 120 (e.g., approaching the object using the recording unit 110, e.g., positioning the recording unit near the object, e.g., at a predetermined position), generating the optical reference image 131, and generating an inspection image 132 of the object using the recording unit with the inspection exposure setting. Reference image 131 and inspection image can be generated, for example, during a single takeoff of the object, i.e., without a stopover.

[0071] Accordingly, embodiments also further comprise a device for adjusting the recording unit 110 for performing an inspection of the object 120, having an evaluation unit configured to obtain the optical reference image 131 generated by the recording unit 110, which at least partially contains the object 120 or 123 to be inspected and an environment 140 of the object. The evaluation unit (e.g., as part of the recording unit or in the form of an external computer, e.g., using a communication unit) is further configured to recognize the object 120 or part 123 within the optical reference image 131; to determine the brightness information 160 based on at least one region of the reference image, which comprises at least a partial region of the object; and to determine an inspection exposure setting for the recording unit 110 using the brightness information 160.

[0072] The images captured by the recording unit can include, for example, individual images or images from a video stream. Optionally, the detection of the object 123' within the optical reference image 131 and / or the determination of the inspection exposure setting for the recording unit 110 can be performed automatically, e.g., based on artificial intelligence.

[0073] Optionally, the object 123 has a characteristic property, i.e., an object characteristic, on the basis of which the inspection exposure setting can be determined. Fig. 2 shows an image of a rotor blade 223 (e.g., as an example of rotor blade 123) against a background 240 in the form of the sky, as an example of a reference image according to embodiments. The rotor blade 223 has a red-colored surface as a characteristic property. The object characteristic can also be, for example, a color combination or, for example, a color pattern.

[0074] Such an object characteristic can be determined automatically, for example, using object recognition or provided to the process as known prior information as input or as a parameter (e.g., stored in a corresponding program). For example, brightness information regarding a "red" color channel, which represents the information about the object characteristic, can be evaluated to determine the inspection exposure setting.

[0075] Thus, for example, the object 223 can be distinguished from the background 240 using the object characteristics and / or object recognition, e.g., recognition of an outline 270, i.e., a border of the object. Accordingly, the inspection exposure setting can be determined based on the brightness information of the reference image selected in this way, e.g., a histogram.

[0076] In other words, the object property can, for example, be a property on the basis of or with respect to which associated brightness information can be evaluated, or with respect to which exposure parameters (i.e. exposure settings) of an associated recording unit can be optimized.

[0077] Furthermore, Fig. 2 shows an example of a result of object recognition according to the invention, wherein the outlines of the rotor blade are highlighted, see border 270. Furthermore, further information can also be determined on the basis of the object recognition, such as, for example, inclination information 280 of the object (e.g., an inclination Angle = 38), i.e., in the example of Fig. 2, e.g., of the rotor blade 223, and / or such as, for example, color information 290 (displayed right at the top of the image), e.g., in the form of information about dominant color values, as shown here, e.g., a display of the dominant red value (number 192) and / or e.g., in the form of information about a color mean value (number 124).

[0078] As discussed above, a first evaluation algorithm, e.g. a KL, can therefore not only recognize the outline 270 of the wing 223, but also the object characteristic “red” and / or the inclination 280.

[0079] Such additional information such as the object characteristics, the outlines 270 and / or the inclination can be used, for example, to align the recording unit with respect to the object, e.g. so that the object or part of the object lies in a predetermined image area.

[0080] In other words, a method according to the invention optionally comprises generating an initial image of the object 223, detecting the object in the initial image, and evaluating the initial image; determining a trigger point for the recording unit to generate the reference image based on the evaluation of the initial image, such that the object is in a predetermined image area of ​​the reference image (e.g., centrally), and positioning the recording unit at (e.g., flying toward) the trigger point and generating the optical reference image of the object at the trigger point using the recording unit.

[0081] Fig. 3 shows schematic views of an object and a background of the object, Fig. 3a), a section of the view from a) in Fig. 3b), as well as schematic plots of brightnesses of color channels for the view from a) in Fig. 3c) and for the view from b) in Fig. 3d), according to embodiments.

[0082] Fig. 3a) shows an object 320 against a background 340. Fig. 3a) can thus represent, for example, a reference image. Fig. 3b) shows a corresponding brightness evaluation of the view from a). The brightness of RGB color channels (red 301, green 302, and blue 303) of the view or image is plotted against the respective portions of the color channels (RGB portion from 0 to 255: from 0 for 0% color saturation and 255 for 100% color saturation). As can be seen, the image thus exhibits a high brightness in a white color range (see peaks 304) and a significantly lower brightness in a gray color range (see peaks 305).

[0083] Fig. 3d) shows a corresponding plot (RGB color channels red 30T, green 302' and blue 303') for the image from b), i.e. for the part 320' of the object 320 and the background section 340', so that here the ratios of the brightness peaks 304' and 305' are changed compared to c).

[0084] Based on the plots in Fig. 3c) and d), it becomes clear that due to the potentially strongly varying image components of object 340 or object part 340' and background 320, 320' and their associated brightnesses, a predictable and, above all, sufficient illumination of an object for inspection purposes is not always readily available.

[0085] According to exemplary embodiments, brightness information, for example information corresponding to the plots in Fig. 3c) and d), for example brightness and color saturation information, for example in the form of histograms, can be determined based on at least one region of the reference image that includes at least a partial region of the object (e.g., 223, 320, 320'). The brightness information can therefore, for example, in particular only include the information in regions 305 and 305' of the plots in Fig. 3c) and d), which are associated with the object or part of the object.

[0086] In Fig. 3a), optional additional information 390, e.g., obtained from object recognition, is also displayed. This optionally includes (shown here as examples): First line: 224 = histogram value of the entire image (here, for example, the entire image 3a)), 166 = histogram of image section 3b) or histogram value of image section 3b); Second line: 220 = histogram of the red channel, or histogram value for the red channel of the entire image, 164 = red channel section 3b), or histogram value for the red channel of image section 3b); Third line: HSV of the entire image; Fourth line: HSV of the red channel.

[0087] As shown in Fig. 2, the object can, for example, have an object characteristic (e.g., "red"), so that one or more brightness thresholds can optionally be determined based on the object characteristic. In other words, a corresponding threshold can, for example, be determined specifically selectively for a red color channel which, for example, due to the object characteristic, has a particularly high information content for the representation quality of the object in the image with regard to an inspection. This threshold is selected, for example, so that a feature of the object to be inspected can be recognized with sufficient certainty. The object feature can, for example, be damage and / or potential damage to the object, such as cracks, holes, bends, chipped paint.

[0088] Such a threshold value can, for example, be determined dynamically in a method according to the invention, e.g. based on machine learning, or, for example, also be stored as a selection parameter, e.g. in a table, e.g. for a selection depending on the object type, additional information or other parameters.

[0089] Based on a threshold value comparison and, for example, an evaluation of a given brightness in comparison to the threshold value, the exposure settings used for the reference image can be adjusted so that an inspection exposure setting can be provided which makes it possible to create an inspection image under the same or at least similar illumination situation of the object as in the reference image, which has a brightness distribution that satisfies the brightness threshold(s). It should be noted again that upper and lower limit values ​​can be used here, for example with regard to avoiding under- or overexposure. Optionally, however, according to embodiments, threshold values ​​(e.g. upper and lower threshold values) for several color channels can also be taken into account. As shown in Fig.As shown in Figure 3, the optical reference image can have several color channels, wherein the brightness information can be color-specific information about a brightness distribution and wherein several color-specific brightness threshold values, for example for the red, green and blue channels, can be determined.

[0090] In particular, these threshold values ​​can be determined for a specific saturation ratio (e.g., between 0 and 255) of the color channels, so that a concrete color channel combination that describes the object, e.g., addresses the object characteristics in the form of the color of the object, can be used to determine the inspection exposure setting with regard to sufficient brightness.

[0091] With reference to Fig. 3, it should be noted that, in general, according to exemplary embodiments, all pixels of the object in the optical reference image and / or a contiguous subset of pixels of the object in the optical reference image, i.e., for example, the object points of the image section in Fig. 3b) and / or a predetermined pattern of pixels of the object in the optical reference image can optionally be used. The pattern can, for example, be selected such that, in particular, regions of the object which are more likely to exhibit damage can be optimally exposed. Furthermore, the number of pixels to be considered can be reduced, in particular with regard to regions of little interest or little significance.

[0092] Optionally, a method according to the invention can be carried out iteratively. For example, the recording unit 110 in Fig. 1 can have a first exposure setting, i.e., for example, a reference exposure setting for generating the reference image 131, during the departure of the object 120. As previously explained, brightness information 160 and corresponding threshold values, e.g., based on an object characteristic, can be derived based on detected pixels of the object 123' in the reference image 131.

[0093] If the brightness information, e.g., peaks 305, 305', associated with the object, satisfies at least one brightness threshold, the reference exposure setting can be determined as the inspection exposure setting. Otherwise, an adjusted reference exposure setting is determined, e.g., as optional features based on the comparison result, e.g., a distance or difference between the desired brightness in the form of the threshold(s) and the actual situation in the reference image, e.g., during takeoff using a processor of the acquisition unit, or using an external data processing unit, e.g., a laptop on the ground.

[0094] Subsequently, an adjusted reference image is generated to simulate the generation of the reference image using the adjusted reference exposure setting. Optionally, the recording unit 110 can also generate another image 130 using the adjusted reference exposure setting.

[0095] The adjusted reference image can now be evaluated for object points with respect to brightness thresholds. If the thresholds are met, an inspection exposure setting is found, for example, with which an image 132 can be taken, enabling reliable object inspection.

[0096] The adjustment of the exposure setting can, for example, generally comprise, according to embodiments, a change in exposure times and / or a change in aperture settings, i.e., for example, a size of the aperture, e.g., a change in the aperture.

[0097] In concrete terms, a process according to the invention can be described as follows:

[0098] 1 . The object to be inspected is detected using object detection / AI in the video stream or single image capture

[0099] 2. Then the brightness of the entire object or only a part of the object is calculated

[0100] 2.1. Determining the color values ​​per pixel in the RGB color space or HSV color space or in the monochrome color space

[0101] 2.2. Determine the dominant color channel (red, green or blue)

[0102] 2.3. Calculate limits according to the dominant color space

[0103] 3. Evaluate the calculated brightness and load the corresponding thresholds. These thresholds were previously defined specifically for the objects.

[0104] 3.1. If brightness is within the limits -> no further changes necessary

[0105] 3.2. If the brightness is outside the limits, continue with step 4. 4. The parameters are adjusted according to the camera's limits (exposure time / aperture).

[0106] 4.1. Over the limit (too bright) -> shorten exposure time (e.g. 1 / 500 -> 1 / 1000) or reduce aperture (e.g. 4.5 -> 5.0)

[0107] 4.2 Below the limit (too dark) -> increase exposure time (e.g. 1 / 1000 -> 1 / 500) or increase aperture (e.g. 5.0 -> 4.5)

[0108] 5. Process starts at 1.

[0109] The brightness adjustment can be done at any time during flight or while the drone is hovering or before each image.

[0110] Reference is made below to Fig. 4. Fig. 4 shows a schematic view of a detection of multiple objects and schematic representations of optical recordings according to embodiments of the invention.

[0111] Fig. 4 shows, in part a), the recording situation from Fig. 1, wherein the recording unit again takes a picture 430 in a backlit situation, but the picture includes both the hub 122 and a rotor blade 123. In the example of Fig. 4, the hub 122 is, for example, partially shaded and the rotor blade 123 is heavily shaded. The remaining system components, i.e., the tower and the two other rotor blades, which are not fully depicted in the picture 430, are not shown in Fig. 4 with regard to their illumination for the sake of simplicity.

[0112] Optionally, a method according to the invention comprises a detection of both objects 122 and 123 in the recording 430. Subsequently, the brightness evaluation can be carried out for each of the two objects separately or jointly.

[0113] For example, after identifying objects 122 and 123, separate brightness histograms can be determined, once for object 122 and once for object 123. Based on a respective evaluation, e.g. as previously discussed in the case of an individual object or object part, respective inspection exposure settings can thus be determined. Subsequently, respective inspection images 432 can be taken for object 122 and 433 for object 123. Images 122" and 123" are then each taken with the individually optimized exposure settings. The schematic representations of images 432 and 433 do not show the exposure situation of the other objects and object parts within the image. The other image parts, which are not in the focus of a subsequent inspection, for example, can be over- or underexposed. In order to avoid such a restriction (for example that several objects orTo circumvent the problem of optimal illumination of the object parts, for example, an image fusion of the images 122", 123" of the objects can be performed to obtain a fusion image 434. For this purpose, the representations 122", 123" can be cut out of their respective recordings and inserted into the reference recording 430 as a replacement for the corresponding image parts.

[0114] In this way, any number of image elements, including the background 140, can be captured with optimized exposure settings and then combined. This allows the inspection to be performed using a single image with each sufficiently exposed object.

[0115] Another possibility is to find a compromise setting for the inspection exposure either based on the individual brightness evaluations of objects 122 and 123 or based on a joint brightness evaluation of objects 122 and 123. Based on this, an inspection image 431 can be taken, which here, as an example, has representations 122' and 123' of both objects, so that they are also sufficiently illuminated for inspection.

[0116] In the following, features of methods according to embodiments are described again in other words:

[0117] According to embodiments, an object (e.g., 122, 123) can be recognized from an image (e.g., 131, 430), e.g., with the aid of an AI. For example, all pixels of this object can be taken into account for calculating the brightness, i.e., for example, for determining brightness information. Subsequently, the exposure values ​​of a camera of a recording unit can be changed based on a defined (or adaptively determined, e.g., with respect to an object characteristic) threshold value. Subsequently, the brightness calculation can be carried out again, i.e., for example, for an image adjusted according to the changed exposure values. This can, for example, be carried out or occur until a required range of values ​​is reached. The object can have any shape in the image space. However, it is also possible, for example, to use not all pixels of the object, but only certain ones, e.g.,from the center of the object (e.g., 100px x 100px) or only pixels that follow a specific pattern (e.g., every third pixel in a row). Optionally, the color spaces can be examined individually to determine dominant influences and make appropriate brightness adjustments (e.g., a dominant red channel requires different thresholds than everything evenly distributed).

[0118] The advantage is that for inspections, the environment outside the object has less influence on the brightness of the image.

[0119] This can be extended to HDR images (High Dynamic Range images). Multiple objects with different brightness levels can be captured individually with the capture unit, e.g., a camera, and then combined (e.g., merged) into a single image. Once the correct setting (e.g., inspection exposure setting) has been found for the object, a setting for the background can also be found. These different images can then be combined again.

[0120] All lists of materials, environmental influences, electrical properties and optical properties listed herein are to be considered exemplary and not exhaustive.

[0121] Although some aspects have been described in the context of a device, it should be understood that these aspects also represent a description of the corresponding method, so that a block or component of a device can also be understood as a corresponding method step or as a feature of a method step. Analogously, aspects described in the context of or as a method step also represent a description of a corresponding block, detail, or feature of a corresponding device. Some or all of the method steps may be performed by (or using) a hardware apparatus, such as a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, some or more of the key method steps may be performed by such an apparatus.

[0122] Depending on specific implementation requirements, embodiments of the invention may be implemented in hardware or software. The implementation may be performed using a digital storage medium, such as a floppy disk, a DVD, a Blu-ray Disc, a CD, a ROM, a PROM, an EPROM, an EEPROM, or a FLASH memory, a hard disk, or other magnetic or optical storage device storing electronically readable control signals that can interact or cooperate with a programmable computer system to perform the respective method. Therefore, the digital storage medium may be computer-readable.

[0123] Some embodiments according to the invention thus comprise a data carrier having electronically readable control signals capable of interacting with a programmable computer system such that one of the methods described herein is carried out.

[0124] In general, embodiments of the present invention may be implemented as a computer program product having a program code, wherein the program code is effective to perform one of the methods when the computer program product is run on a computer.

[0125] The program code can, for example, also be stored on a machine-readable medium.

[0126] Other embodiments include the computer program for performing one of the methods described herein, wherein the computer program is stored on a machine-readable carrier.

[0127] In other words, an embodiment of the method according to the invention is thus a computer program which has a program code for carrying out one of the methods described herein when the computer program runs on a computer.

[0128] A further embodiment of the method according to the invention is thus a data carrier (or a digital storage medium or a computer-readable medium) on which the computer program for performing one of the methods described herein is recorded. The data carrier, the digital storage medium, or the computer-readable medium is typically physical and / or non-perishable or non-transient.

[0129] A further embodiment of the method according to the invention is thus a data stream or a sequence of signals that represents the computer program for carrying out one of the methods described herein. The data stream or the sequence of signals can be configured, for example, to be transferred via a data communication connection, for example, via the Internet.

[0130] A further embodiment comprises a processing device, for example a computer or a programmable logic device, which is configured or adapted to carry out one of the methods described herein.

[0131] A further embodiment comprises a computer on which the computer program for performing one of the methods described herein is installed.

[0132] A further embodiment according to the invention comprises a device or system designed to transmit a computer program for performing at least one of the methods described herein to a recipient. The transmission can be electronic or optical, for example. The recipient can be, for example, a computer, a mobile device, a storage device, or a similar device. The device or system can, for example, comprise a file server for transmitting the computer program to the recipient.

[0133] In some embodiments, a programmable logic device (e.g., a field-programmable gate array, an FPGA) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field-programmable gate array may cooperate with a microprocessor to perform any of the methods described herein. In general, in some embodiments, the methods are performed by any hardware device. This may be general-purpose hardware such as a computer processor (CPU) or method-specific hardware such as an ASIC.

[0134] The devices described herein may be implemented, for example, using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.

[0135] The devices described herein, or any components of the devices described herein, may be implemented at least partially in hardware and / or in software (computer program). The methods described herein may be implemented, for example, using a hardware device, or using a computer, or using a combination of a hardware device and a computer. The methods described herein, or any components of the methods described herein, may be executed at least partially by hardware and / or by software.

[0136] The above-described embodiments are merely illustrative of the principles of the present invention. It is understood that modifications and variations of the arrangements and details described herein will be apparent to others skilled in the art. Therefore, it is intended that the invention be limited only by the scope of the following claims and not by the specific details presented in the description and explanation of the embodiments herein.

Claims

Patent claims 1. A method for setting a recording unit (110) for carrying out an inspection of an object (120, 121, 122, 123, 223, 320), comprising the following steps: Providing an optical reference image (131, 430) generated by the recording unit, which contains at least partially the object to be inspected and an environment (140, 240, 340) of the object; Detecting the object within the optical reference image; Determining brightness information (160) based on at least one area of the reference recording which has at least a partial area of the object; Determining an inspection exposure setting for the acquisition unit using the brightness information.

2. The method according to claim 1, wherein the object (120, 121, 122, 123, 223, 320) has an optically detectable object characteristic, such as a color, a pattern, and / or a color pattern; and wherein determining the inspection exposure setting for the recording unit (110) is performed using the brightness information (160) and the object characteristic.

3. The method according to claim 2, wherein the method further comprises the following step: Recognition of the optically detectable object characteristics based on the optical reference image (131, 430).

4. Method according to one of claims 2 or 3, wherein the brightness information (160) comprises information about a brightness distribution of the at least one area of the reference image (131, 430); and the method further comprising the following steps: Determining at least one brightness threshold based on the object characteristics; and Determining the inspection exposure setting for the recording unit (110) using the brightness information and the at least one brightness threshold value, so that an inspection image of the object (120, 121, 122, 123, 223, 320) which is taken by means of the recording unit with the Inspection exposure setting under an identical or at least similar illumination situation of the object as in the reference image, has a brightness distribution which satisfies the brightness threshold.

5. The method according to claim 4, wherein the method is a method for adjusting a recording unit for performing an inspection of the object (120, 121, 122, 123, 223, 320) with respect to an object feature to be optically recognized; and wherein the determination of the at least one brightness threshold value is carried out based on the object feature to be recognized such that the object feature to be recognized is recognizable from the inspection image of the object.

6. The method according to claim 5, wherein the object feature is damage and / or potential damage to the object (120, 121, 122, 123, 223, 320).

7. The method according to any one of claims 4 to 6, wherein the optically detectable object characteristic is a color, a color pattern, and / or a color distribution of the object (120, 121, 122, 123, 223, 320); wherein the optical reference image (131, 430) has a plurality of color channels; and wherein the brightness information (160) is color-specific information about a brightness distribution; and wherein determining the at least one brightness threshold comprises determining one or more color-specific brightness thresholds.

8. Method according to one of the preceding claims, wherein all pixels of the object (120, 121, 122, 123, 223, 320) in the optical reference image (131, 430) and / or a contiguous subset of pixels of the object in the optical reference image; and / or a predetermined pattern of pixels of the object in the optical reference image; are used to determine the brightness information (160).

9. Method according to one of the preceding claims, wherein the optical reference recording (131, 430) is on a Reference exposure setting of the recording unit; wherein determining the inspection exposure setting for the recording unit (110) is performed using the brightness information (160) and at least one brightness threshold, and further comprising the following steps: Comparing the brightness information with the at least one brightness threshold, wherein, if the brightness information satisfies the at least one brightness threshold, the reference exposure setting is determined as the inspection exposure setting, and wherein, if the brightness information does not satisfy the at least one brightness threshold, the following steps Determining an adjusted reference exposure setting using the comparison; Creating an adjusted reference exposure to simulate creating the reference exposure using the adjusted reference exposure setting; Determining an adjusted brightness information based on at least one area of the adjusted reference image, which has at least a partial area of the object (120, 121, 122, 123, 223, 320); Comparing the adjusted brightness information with the at least one brightness threshold; carried out iteratively until the adjusted brightness information satisfies the at least one brightness threshold, wherein the adjusted reference exposure setting by means of which the brightness threshold is satisfied is then determined as the inspection exposure setting.

10. The method according to any one of the preceding claims, wherein the object is a first object (123) and wherein the optical reference image (430) comprises a second object (122); and wherein the method further comprises the following steps: Detecting the second object within the optical reference image; Determining a second brightness information based on at least one area of the reference recording which has at least a partial area of the second object; Determining a second inspection exposure setting for the capture unit (110) using the second brightness information; Creating a first optical inspection image (432) of the first object by means of the recording unit with the inspection exposure setting; and Generating a second optical inspection image (433) of the second object by means of the recording unit with the second inspection exposure setting.

11. The method according to claim 10, wherein the method further comprises the step: Merging the first and second optical inspection images.

12. The method according to any one of the preceding claims, wherein the object is a first object (123) and wherein the optical reference image (430) comprises a second object (122); and wherein the method further comprises the following steps: Detecting the second object within the optical reference image; Determining a common brightness information based on the at least one area of the reference image which has at least a partial area of the first object and based on at least one area of the reference image which has at least a partial area of the second object, and determining a common inspection exposure setting for the recording unit (110) using the common brightness information; or Determining a second brightness information item based on at least one area of the reference image which includes at least a partial area of the second object, and determining a common inspection exposure setting for the recording unit using the first and second brightness information items; and Generating an optical inspection image (431) of the first and second objects by means of the recording unit with the common inspection exposure setting.

13. A method for inspecting an object (120, 121, 122, 123, 223, 320) comprising a method according to any one of the preceding claims, wherein the recording unit (110) is a movable recording unit; and wherein the method further comprises the steps: Moving the recording unit to the object; Generating the optical reference image (131, 430); Creating an inspection image of the object using the acquisition unit with the inspection exposure setting.

14. The method according to claim 13, wherein the method further comprises the steps of: Creating an initial image of the object (120, 121, 122, 123, 223, 320); Detecting the object in the initial image and evaluating the initial image; Determining a trigger point for the recording unit (110) for generating the reference recording (131, 430) based on the evaluation of the initial recording, so that the object is in a predetermined image area of the reference recording; and Positioning the recording unit at the trigger point and creating the optical reference image of the object at the trigger point using the recording unit.

15. Method according to one of the preceding claims, wherein the detection of the object (120, 121, 122, 123, 223, 320) within the optical reference recording (131, 430) and / or the determination of the inspection exposure setting for the recording unit is carried out automatically, e.g. based on artificial intelligence.

16. Method according to one of the preceding claims, wherein the detection of the object within the optical reference recording (131, 430) is carried out by means of a first neural network, and wherein the determination of the inspection exposure setting for the recording unit is carried out under Use of the brightness information (160) is carried out by means of a second neural network; or wherein the detection of the object within the optical reference image (131, 430) is carried out by means of a neural network, and wherein the determination of the inspection exposure setting for the image acquisition unit is carried out using the brightness information (160) by means of a computer vision algorithm;or wherein the detection of the object within the optical reference image (131, 430) is carried out by means of a computer vision algorithm, and wherein the determination of the inspection exposure setting for the recording unit is carried out using the brightness information (160) by means of a neural network, or wherein the detection of the object within the optical reference image (131, 430) is carried out by means of a first computer vision algorithm, and wherein the determination of the inspection exposure setting for the recording unit is carried out using the brightness information (160) by means of a second computer vision algorithm.; 17. Method according to one of the preceding claims, wherein the object (120, 121, 122, 123, 223, 320) is a wind turbine or a part of a wind turbine, such as a rotor blade (123), a rotor blade hub (122), a rotor blade flange, a tower (121) and / or a nacelle.

18. The method according to claim 17, wherein the method is a method for inspecting a component of a wind turbine with respect to optically detectable damage and / or potential damage; wherein the recording unit (110) is a drone; and wherein the method comprises the following steps: Approaching the component of the wind turbine using the recording unit; Generating an optical reference image (131, 430) of the component of the wind turbine by means of the recording unit; Automated detection of the wind turbine component within the optical reference image; Determining a brightness distribution based on a subset of sample points of the reference image that includes the detected component of the wind turbine; Determining an inspection exposure setting for the recording unit using the brightness distribution, so that damage and / or potential damage to the component of the wind turbine can be optically detected in an inspection image of the component of the wind turbine, which is generated by means of the recording unit with the inspection exposure setting under an identical or at least similar illumination situation of the component of the wind turbine as in the reference image.

19. A device for adjusting a recording unit (110) for carrying out an inspection of an object (120, 121, 122, 123, 223, 320), the device having the following features: an evaluation unit configured to obtain an optical reference recording (131, 430) generated by a recording unit, which at least partially contains the object to be inspected and an environment (140, 240, 340) of the object; to detect the object within the optical reference recording; to determine brightness information (160) based on at least one region of the reference recording, which has at least a partial region of the object; and to determine an inspection exposure setting for the acquisition unit using the brightness information.

20. The device according to claim 19, further comprising: the recording unit (110) which is designed to generate the optical reference recording (131, 430).

21. The device according to claim 20, wherein the recording unit (110) is a drone for flying over the object (120, 121, 122, 123, 223, 320); and wherein the recording unit is configured to perform the reference recording (131, 430) and an inspection recording of the object using the inspection exposure setting without intermediate landing.

22. The device according to claim 21, wherein the recording unit (110) comprises the evaluation unit; or wherein the device comprises a communication unit configured to provide the evaluation unit with information about the optical reference recording (131, 430) from the recording unit and to provide the recording unit with information about the inspection exposure setting from the evaluation unit.

23. A computer program having a program code for carrying out the method according to one of claims 1 to 18 when the program runs on a computer.

Images